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Executive  Summary 

Given  the  increased  focus  on  assessment  and  accountability  since  the  1990s,  the  National 
Endowment  for  the  Arts  (NEA)  identified  the  need  to  capture  the  current  status  of  arts  assessment. 
In  2005,  the  NEA  began  requiring  a narrative  statement  of  assessment  practices  to  apply  for  arts 
education  funds.  Project  applicants  needed  to  explain  their  assessments  methods  and  types  of  tools 
used  to  measure  student  knowledge  and  skills.  Through  several  grant  cycles,  it  became  clear  to 
NEA  staff  that  applicants  did  not  necessarily  differentiate  between  program  evaluation  and 
assessment  of  student  learning.  As  such,  the  NEA  commissioned  WestEd  to  examine  current 
trends,  promising  techniques,  and  successful  practices  being  used  to  assess  student  learning  in  the 
arts  throughout  the  country,  as  well  as  identify  potential  areas  in  which  arts  assessment  could  be 
improved.  Although  the  original  intent  of  the  study  was  to  identify  strong  models  of  assessment 
practices  that  could  serve  as  examples  for  possible  replication,  the  study  found  that  such  models 
were  not  available  and  are  in  fact  a need  ot  the  field.  Thus,  this  report  provides  a description  of  the 
current  state  of  arts  assessment,  including  a review  of  the  high-quality  literature  available,  common 
practices  being  used  to  assess  student  learning,  and  needs  of  the  field  to  improve  arts  assessment. 

Through  this  study  — the  first  of  its  kind  — the  NEA  and  WestEd  sought  to  collect,  analyze,  and 
report  on  information  about  current  practices  and  the  needs  of  the  field  related  to  the  assessment  of 
K-12  student  learning  in  the  arts.  Understanding  the  assessment  experiences  and  practices  of  arts 
education  stakeholders,  including  their  needs,  is  one  step  toward  helping  improve  student 
assessment  in  the  arts.  The  goals  of  the  current  study  included  identifying: 

• Available  resources,  tools,  and  documentation  related  to  the  assessment  of  student 
learning  in  the  arts 

• Current  experiences  and  practices  in  assessing  student  knowledge  in  the  arts 

• Current  experiences  and  practices  in  assessing  student  skills  in  the  arts 

• Trends  in  locating  and  using  assessment  tools 

• Needs  of  the  field  to  improve  the  assessment  of  student  learning  in  the  arts 


STUDY  METHODOLOGY 


Three  data  collection  methods  were  used  to  address  the  study  goals:  (1)  conversations  with  expert 
consultants  in  the  field;  (2)  a review  of  arts  assessment  literature,  including  tools,  resources, 
informational  materials,  and  research  reports;  and  (3)  a nationwide  survey  administered  to 
policymakers,  educators,  arts  and  cultural  organization  staff,  and  researchers. 

WestEd  collaborated  with  the  NEA  to  identify  key  leaders  in  the  field  of  arts  assessment,  and 
WestEd  researchers  spoke  with  21  expert  consultants  throughout  the  country',  representing  a range 
of  art  forms,  stakeholder  groups,  geographic  locations,  and  backgrounds.  The  consultants  shared 
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their  perspectives  on  current  assessment  trends  and  practices,  identified  additional  sources  of 
literature  for  review,  and  provided  input  into  the  project  survey. 

The  literature  review  component  was  intended  to  identify  documents  about  assessing  student 
learning  in  the  arts,  such  as  assessment  tools,  how-to  resources,  technical  and  research  reports,  and 
other  informational  documents.  The  literature  search  process  cast  a wide  net  that  captured  more 
than  1,000  pieces  of  literature.  During  the  review  process,  documents  were  rated  for  relevance  to 
the  study  and  quality  to  ensure  only  highly  relevant  and  high-quality1  materials  were  analyzed  tor  the 
study. 

The  nationwide  survey  was  designed  to  collect  information  directly  from  policymakers,  educators, 
arts  and  cultural  organization  staff,  and  arts  researchers  about:  (1)  current  practices  in  assessing 
student  learning  in  the  arts;  (2)  the  types  of  assessments  being  used  to  measure  student  learning;  (3) 
how  assessment  tools  are  identified/developed  and  used  in  the  field;  and  (4)  what  assistance  the  field 
needs  to  improve  assessment  of  student  learning  in  the  arts.  The  survey  was  completed  by  nearly 
3,750  people;  data  were  used  from  3,377  respondents  representing  school  staff,  arts/cultural 
organization  staff,  researchers/evaluators,  district  staff,  state/county  arts  council  staff,  and 
state/county  office  of  education  staff  (Exhibit  ES  I). 


Exhibit  ES-1  — Primary  Role  of  Respondents  included  in  the  Study 


Number  of 
Respondents 

Percent 

School  Staff 

2,079 

61.6 

Arts/Cultural  Organization  Staff 

840 

24.9 

Arts  Researchers/Evaluators 

205 

6.1 

District  Staff 

90 

2.7 

State/County  Arts  Council  Staff 

84 

2.5 

State/County  Office  of  Education  Staff 

79 

2.3 

Total 

3,377 

100 

* Quality  was  gauged  entirely  in  tenns  of  the  content  of  the  information.  Quality  of  content  was  based  on  the 
information  provided  for  the  type  of  document  and  the  appropriateness  to  the  intended  audience.  For  example,  a higher 
quality  assessment  tool  would  have  reliability-  and  validity  informadon  available,  while  a higher  quality  resource  would 
have  clear  and  easy  to  follow  instructions.  The  guidelines  used  to  gauge  quality  for  each  type  of  document  are  presented 
in  Appendix  A. 
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FINDINGS 


There  is  a lack  of  publicly  available  high-quality  assessment  tools,  informational 
documents,  how-to  resources,  and  technical  reports  related  to  K-12  student  learning  in  the 
arts. 


• Of  727  individual  items  reviewed,  only  148  (20.4%)  were  both  relevant  to  the  study 
and  of  high  quality. 

• The  majority  of  high-quality  assessment  tools  focused  on  visual  arts  and/or  music. 

Few  research  and  technical  reports  are  publicly  available. 

• Although  evaluation  reports  for  arts  projects  are  required  by  many  funders,  they  are 
generally  not  being  released  publicly. 

• Much  of  the  available  research  literature  focuses  on  learning  through  the  arts  rather 
than  learning  in  the  arts. 

A lack  of  clarity  exists  regarding  the  difference  between  arts  knowledge  and  arts  skills. 

• Survey  respondents  reported  measuring  student  knowledge  with  methods  more 
appropriate  for  measuring  skills  and  vice  versa  (e.g.,  use  of  a paper/pencil  test  to 
measure  student  skills).  This  calls  into  question  the  validity  of  assessments  designed 
and  used  by  those  who  may  not  fully  understand  the  difference. 

Survey  respondents  use  a variety  of  assessment  tools  to  collect  data  for  multiple  purposes. 

• All  groups  of  respondents  reported  using  many  different  types  of  skills  assessment 
tools,  including  rubrics,  observation  protocols,  portfolio  reviews,  and  performance- 
based  assessments. 

• The  majority'  of  survey  respondents  reported  that  the  tool  they  found  most  useful 
was  created  by  a teacher  or  teaching  artist. 

• Reasons  for  collecting  data  included  formative  feedback,  program  evaluation,  and 
district/school  accountability.  School  staff  most  often  reported  using  data  for 
student  grades,  while  arts  and  cultural  organizations  and  state/ county  arts  council 
staff  were  significantly  more  likely  to  collect  data  as  a funding  requirement. 

The  majority  of  high-quality,  publicly  available  assessment  tools  are  created  by  large-scale 

testing  agencies  and  state  education  agencies. 

• In  general,  assessments  created  by  these  larger  agencies  scored  higher  for  quality  than 
did  assessments  created  by  individuals  or  smaller  organizations. 
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A need  exists  for  a single,  comprehensive  clearinghouse  for  tools,  information,  and 
resources  focused  on  assessing  student  knowledge  and  skills  in  the  arts. 

• Existing  documents  identified  during  the  literature  review  process  were  scattered 
across  many  websites,  journal  articles,  books,  and  other  documents.  High-quality 
materials  were  often  mixed  in  with  low-quality  materials,  making  them  even  more 
difficult  to  locate  and  identify. 


• Overall,  the  quality  of  assessment  tools  found  on  the  web  was  low  - yet  more  than 
three-quarters  of  survey  respondents  reported  they  use  Internet  search  engines  to 
look  for  assessment  tools,  often  with  little  success. 

• Survey  respondents  identified  the  need  for  exemplar  tools  (e.g.,  specific  assessment 
tools,  examples,  and  item  banks)  and  models  of  successful  assessment  practices  to 
learn  from  and  replicate. 


More  than  three-quarters  of  survey  respondents  also  reported  they  would  create  a 
new  assessment  tool  if  they  needed  one;  however,  locally  developed  tools  tended  to 
receive  lower  ratings  for  quality  (potentially  stemming  from  a lack  of  clarity  between 
knowledge  and  skills  and  a lack  of  understanding  about  what  is  and  is  not  a rubric). 


There  is  a need  for  professional  development  related  to  arts  assessment. 

- cM-y"?  or  a»(»r©& S ? 

• Overall,  the  arts  education  field  is  eager  to  assess  student  learning.  However,  the 
field  needs  guidance  and  assistance  to  implement  high-quality  assessment  practices. 

• Findings  from  the  literature  review  and  survey  responses  indicated  a need  for 
training  on  the  difference  between  assessing  student  knowledge  and  skills  in  the  arts. 

• Findings  from  the  literature  review  and  survey  responses  indicated  a need  for 
training  regarding  rubrics,  particularly  what  constitutes  a rubric,  how  it  is  properly 
used,  and  what  components  are  necessary  to  develop  or  select  a high-quality  rubric. 

• More  than  half  of  survey  respondents  reported  receiving  training  on  arts  assessment 
via  professional  development  workshops  or  conferences,  whereas  fewer  than  half  of 
all  respondents,  including  fewer  than  one-quarter  for  some  respondent  groups, 
reported  receiving  undergraduate-  or  graduate-level  training  on  assessing  student 
learning. 

• Respondents  reported  needing  additional  training  on  topics  such  as:  locating  and 
identifying  valid  assessment  tools,  using  rubrics  and  other  assessment  methods,  and 
using  assessment  to  demonstrate  the  importance  of  the  arts. 


Survey  respondents  reported  needs  of  the  field  around  four  main  categories  — guidance, 
trained  professionals,  making  the  case,  and  additional  needs. 


• Guidance:  a clear  framework  aligning  standards,  curriculum,  and  instruction;  access 
to  exemplar  tools;  models  of  assessment  practice;  resources;  and  professional 
learning  communities  (PLCs)  to  share  knowledge  and  ask  questions. 
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• Trained  professionals:  professional  development,  university  training,  and  certification 
programs  in  all  art  forms  to  improve  instruction  and  assessment  in  the  arts. 

• Making  the  case:  demonstrating  the  value  of  the  arts,  including  having  research  to 
show  the  impact  of  arts  education;  garnering  support  from  school  and  district 
leaders;  and  implementing  statewide  or  high-stakes  testing  as  a method  to  increase 
the  perceived  importance  of  the  arts  among  policymakers  and  other  stakeholders. 

• Additional  needs:  funding,  time,  technology,  meeting  the  needs  of  diverse  students, 
overcoming  anti-assessment  sentiment,  and  addressing  the  “subjective  myth”  that 
the  arts  are  subjective  and  cannot  be  assessed  objectively. 

Exhibit  ES-2  — Top  Ten  Needs  of  the  Field  as  Expressed  by  Survey  Respondents 


District 

Staff 

School 

Staff 

State/ 
County 
Office  of 
Education 

State/ 
County  Arts 
Council  Staff 

Arts/Cultural 

Organization 

Staff 

Arts 

Researchers 

and 

Evaluators 

% 

% 

% 

% 

% 

% 

Exemplar  tools 

40.0 

24.3 

25.8 

27.0 

32.8 

26.9 

Framework 

26.3 

38.5 

39.4 

36.5 

20.6 

36.5 

Professional  development 

26.3 

20.8 

33.3 

44.6 

23.0 

22.2 

Alternative  assessments 

21.3 

18.9 

28.8 

16.2 

19.3 

24.0 

Models 

17.5 

10.6 

15.2 

18.9 

12.9 

16.2 

Professional  learning 
communities 

13.8 

10.9 

9.1 

9.5 

8.9 

15.0 

Value  of  arts 

10.0 

11.4 

16.7 

13.5 

16.2 

18.6 

Research 

8.8 

5.4 

9.1 

21.6 

24.1 

23.4 

Funding 

6.3 

13.3 

27.3 

35.1 

29.0 

16.8 

Resources 

3.8 

8.4 

10.6 

9.5 

13.6 

3.6 

Distnct  staff  n = 80,  school  staff  n = 1715,  office  of  education  n = 66,  arts  council  staff  n — 74,  arts/cultural 
organization  staff  n = 618,  researchers/evaluators  n = 167 

Note  — Alternative  assessments  are  defined  as  assessments  that  provide  students  and  opportunity  to  create  a response  to  a 
question,  task,  or  assignment  (e.g.,  performance,  oral  presentation,  exhibition),  as  opposed  to  a traditional  assessment  in 
which  students  choose  a response  option  such  as  true-false,  multiple-choice,  or  matching. 

Overall,  the  arts  education  field  is  eager  to  assess  student  learning  — survey  respondents  reported 
using  a variety  of  assessment  tools  to  collect  data  for  multiple  purposes.  1 lowever,  the  field  needs 
further  guidance  and  assistance  to  implement  high-quality  assessment  practices.  A clearinghouse  is 
needed  to  address  the  lack  of  publicly  available  high-quality  assessment  tools,  informational 
documents,  how-to  resources,  and  technical  reports  related  to  student  learning  in  the  arts.  There  is 
also  a strong  need  for  professional  development,  both  to  address  misconceptions  and  to  improve 
the  assessment  of  student  learning  in  the  arts.  In  particular,  professional  development  is  needed  to 
clarify  the  distinction  between  knowledge  and  skills  in  the  arts;  to  clearly  define  rubrics,  how  to 
locate/develop  them,  and  how  they  are  used  to  assess  learning;  and  to  dispel  the  myth  that  the  arts 
are  subjective  and  thus  not  able  to  be  objectively  assessed.  The  following  recommendations  to 
improve  the  assessment  of  student  learning  in  the  arts  were  derived  based  on  the  literature  review 
and  findings  from  the  nationwide  survey. 
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RECOMMENDATIONS 


Assemble  a national  advisory  committee  to  bridge  assets  and  come  to  consensus  on  how  to 
improve  arts  assessment. 

• Include  members  from  all  stakeholder  groups  (e.g.,  teachers,  teaching  artists,  district 
staff,  policymakers,  arts/cultural  organization  staff,  researchers). 

• Tasks  for  the  advisory  committee  should  include  conceptualizing  an  arts-assessment 
clearinghouse,  setting  a national  research  agenda,  and  prioritizing  professional 
development  topics  for  the  field. 

Create  an  online  clearinghouse  for  high-quality  arts  assessment  materials. 

• Develop  a one-stop  shop  where  teachers,  teaching  artists,  practitioners,  and 
researchers  can  access  reliable  and  valid  assessment  tools,  helpful  how-to  resources, 
high-quality  research  and  evaluation  reports,  and  relevant  informational  documents 
pertaining  to  arts  assessment. 

• Ensure  high-quality,  vetted  measures  and  resources  are  available,  leading  to 
improved  validity  in  arts  assessment.  Agencies  and  organizations  looking  to  develop 
assessment  tools  must  be  willing  to  commit  the  time,  money,  and  resources 
necessary  to  design  high-quality  measures. 

• Features  of  a high-quality  clearinghouse  should  include: 

• High-quality  informational  documents  on  myriad  topics,  including  the  distinction 
between  knowledge  and  skills  in  the  arts  and  glossaries  of  important 
evaluation-  and  assessment-related  terms. 

• High-quality  how-to  resources  on  myriad  topics,  including  how  to 
locate/identify/develop  valid  and  reliable  assessment  tools,  especially  rubrics. 

• High-quality  exemplar  assessment  too/s,  including  measures  that  are  appropriate 
for  varying  art  forms  and  grade  levels,  address  the  needs  of  appropriate 
audiences  (e.g.,  classroom  teachers,  teaching  artists),  offer  time-sensitive 
options,  are  easily  adaptable  with  instruction  on  how  the  tool  can  be 
modified,  and  provide  data  that  can  contribute  to  larger  research  efforts. 

• While  the  clearinghouse  could  take  years  to  fully  establish,  in  the  short-term  create  a 
website  with  informational  materials  and/or  professional  development  programs  on 
high-priority  topics  like  the  difference  between  knowledge  and  skills  in  the  arts  and 
how  to  identify,  develop,  and  use  a quality  rubric. 

• First  steps  should  be  taken  on  the  NEA  website  to  maximize  the  impact  of  pre- 
existing traffic  flow  since  survey  respondents  indicated  they  already  visit  the  site  in 
search  of  assessment-related  tools  and  resources. 
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Establish  online  professional  learning  communities  (PLCs). 

• PLCs  could  be  part  of  the  clearinghouse  or  a separate  entity. 

• Maintain  special  communities  for  different  stakeholder  groups  (e.g.,  policymakers, 
teachers,  teaching  artists,  arts/cultural  organization  staff,  researchers). 

• Establish  communities  across  content  or  topic  areas,  such  as  a location  where 
participants  can  upload  their  assessment  tool  and  receive  constructive  feedback,  or 
share  ideas  on  meeting  the  needs  of  diverse  students. 

Increase  professional  development  offered  in  the  area  of  arts  assessment. 

• Provide  current  information  aimed  at  developing  common  understandings,  sharing 
successful  practices,  and  building  the  knowledge  and  skills  needed  to  implement 
assessment  in  the  arts. 

• Tailor  professional  development  to  arts  education  audiences,  such  as  separate  tracks 
for  teachers,  teaching  artists,  policymakers/ administrators,  researchers,  and 
arts/cultural  organization  staff  Professional  development  providers  should  be 
selected  based  on  the  specific  needs  of  each  group. 

• Establish  criteria  for  high-quality  professional  development  that  encompass  webinars 
and  regional  trainings/conferences. 

• Address  specific  need  for  professional  development  related  to  rubrics,  including  how 
to  accurately  define  and  identify  them,  how  to  select  a high-quality  rubric,  and  how 
to  modify  or  develop  a rubric  to  meet  assessment  needs. 

• Address  specific  need  for  professional  development  related  to  recognizing  high- 
quality  assessment  materials  — particularly  tools  — since  more  than  three-quarters  of 
survey  respondents  reported  using  the  Internet  to  search  for  measures. 

• Partnerships  among  offices  of  education,  arts  councils,  universities,  and  researchers 
can  be  used  to  bridge  assets  and  provide  comprehensive  professional  development 
programs. 

Develop  a national  arts  assessment  research  agenda  and  prioritize  dissemination  of  tools 
and  reports. 

• Additional  research  is  needed  on  a variety  of  topics  (e.g.,  identifying  models  of 
successful  practice  in  various  settings,  demonstrating  how  learning  in  the  arts  is 
beneficial  to  students). 

• Priorities  should  be  defined  by  the  advisory  committee  and  made  public  so 
researchers  can  respond. 

• Establish  a website  for  vetted,  high-quality  research  offering  both  brief  summaries 
and  full  reports.  Encourage  researchers  to  publish  findings  on  this  site  and  in  other 
locations  accessible  to  the  arts  education  community. 
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• Assist  funders  and  others  to  make  evaluation  tools  and  reports  public.  Address  the 
issues  of  negative  results  and  participant  confidentiality. 

Stop  re-creating  the  wheel. 

• Take  action  to  reduce  duplication  and  maximize  efficacy. 

• Establish  a clearinghouse  for  high-quality  arts  assessment  tools  and  resources, 
allowing  practitioners  to  easily  locate  high-quality,  vetted  materials  that  meet  their 
needs. 

• Funders  with  constrained  resources  should  work  together  to  determine  criteria  and 
methods  for  grantee  reporting,  streamlining  the  process  lor  recipients  of  multiple- 
funding  sources  and  possibly  allowing  lor  some  comparative  data. 

• Push  to  make  evaluadon  reports  publicly  available,  allowing  arts  organizations  and 
researchers  to  learn  what  methodologies  have  been  used  in  the  past  and  with  what 
results,  identify  and  build  upon  best  practices,  and  avoid  pitfalls  through  lessons 
learned. 

Much  is  needed  to  improve  the  assessment  of  student  learning  in  the  arts.  Establishing  a national 
clearinghouse  for  high-quality  assessment  tools,  informational  documents,  how-to  resources,  and 
research  and  evaluation  reports  is  a key  priority  and  would  allow  teachers,  teaching  artists, 
policymakers,  practitioners,  researchers,  and  other  interested  parties  to  easily  access  vetted  materials. 
Professional  learning  communities  should  be  established  to  support  arts  educators  in  their 
assessment  efforts  and  allow  for  sharing  of  both  questions  and  best  practices.  In  addition,  extensive 
professional  development  needs  to  be  offered  to  targeted  audiences  on  a wide  variety  of  topics  in 
order  to  increase  the  quality  and  validity  of  arts  assessment.  Three  specific  areas  of  need  are:  (1) 
developing  valid  and  reliable  rubrics;  (2)  recognizing  high-quality  assessment  materials;  and  (3) 
understanding  the  difference  between  arts  knowledge  and  skills,  particularly  with  regard  to 
appropriate  assessment  methods.  A representative  advisory  committee  should  be  tasked  with  setting 
a national  research  agenda,  prioritizing  professional  development  topics,  and  conceptualizing  the 
clearinghouse.  While  not  exhaustive,  these  recommendations  set  the  stage  for  significant 
improvements  in  the  quality-,  efficiency,  cohesion,  and  usefulness  of  student  assessment  in  the  arts, 
in  order  to  ultimately  improve  student  learning. 
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Study  Overview 

With  the  creation  of  national  standards  for  arts  learning  in  the  1990s,  assessment  of  student  learning 
in  the  arts  has  become  critical  for  providers  of  arts  instruction  throughout  the  country.  Formative 
assessment  can  be  used  to  provide  constructive  feedback  to  students  and  adjust  instruction,  while 
summative  assessment  can  provide  measures  of  progress,  benchmarks  for  learning  and  growth,  or 
evaluations  of  student  work.  The  assessment  of  student  learning  can  also  contribute  important 
information  and  data  that  may  feed  a larger  program  evaluation  or  research  study.  Further,  the 
tracking  of  longitudinal  data  allows  for  comparisons  over  time,  identification  of  trends  or  patterns, 
and  can  help  demonstrate  the  impact  of  programmatic  changes. 

In  2005,  the  National  Endowment  for  the  Arts  (NEA)  began  requiring  a narrative  statement  of 
assessment  practices  to  apply  for  arts  educadon  funds.  Through  the  review  of  applications  across 
multiple  funding  cycles,  the  NEA  became  aware  of  inconsistent  availability  and  use  of  assessment 
tools,  practices,  and  terminology.  In  response  to  feedback  from  panelists  reviewing  more  than  700 
grants  in  more  than  seven  disciplines  per  year,  the  NEA  set  out  to  understand  current  assessment 
trends  and  needs.  As  such,  the  NEA  commissioned  WestEd  to  examine  the  current  status  of  arts 
assessment  to  capture  trends  and  practices  as  well  as  identify  potential  areas  in  which  the  assessment 
of  student  learning  in  the  arts  could  be  improved. 

Through  this  study  — the  first  of  its  kind  - the  NEA  and  WestFld  sought  to  collect,  analyze,  and 
report  on  information  about  current  practices  and  the  needs  of  the  field  related  to  the  assessment  of 
K-12  student  learning  in  the  arts.  Understanding  the  assessment  experiences  and  practices  of  arts 
education  stakeholders,  including  their  needs,  is  one  step  toward  helping  improve  student 
assessment  in  the  arts.  The  goals  of  the  current  study  included  identifying: 

• Available  resources,  tools,  and  documentation  related  to  the  assessment  of  student 
learning  in  the  arts 

• Current  experiences  and  practices  in  assessing  student  knowledge  in  the  arts 

• Current  experiences  and  practices  in  assessing  student  skills  in  the  arts 

• Trends  in  locating  and  using  assessment  tools 

• Needs  of  the  field  to  improve  the  assessment  of  student  learning  in  the  arts 

Three  data  collection  methods  were  used  to  address  the  study  goals  — (1)  conversations  with  expert 
consultants  in  the  field;  (2)  a review  of  arts  assessment  literature,  including  tools,  resources, 
informational  materials,  and  research  reports;  and  (3)  a nationwide  survey  administered  to 
policymakers,  educators,  arts  and  cultural  organization  staff,  and  researchers. 

WestEd  collaborated  with  the  NEA  to  identify  key  leaders  in  the  field  of  arts  assessment,  and 
WestEd  researchers  spoke  with  21  expert  consultants  throughout  the  country.  The  consultants 
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shared  their  perspectives  on  current  assessment  trends  and  practices,  identified  additional  sources  of 
literature  for  review,  and  provided  input  into  the  project  survey. 

The  literature  review  component  was  intended  to  identify  documents  about  assessing  student 
learning  in  the  arts,  such  as  assessment  tools,  how-to  resources,  technical  and  research  reports,  and 
other  informational  documents.  The  literature  search  process  cast  a wide  net  that  captured  more 
than  1,000  pieces  of  literature.  During  the  review  process,  documents  were  rated  for  relevance  to 
the  study  and  quality  to  ensure  only  highly  relevant  and  high-quality  materials  were  analyzed  for  the 
study. 

The  nationwide  survey  was  designed  to  collect  information  directly  from  policymakers,  educators, 
arts  and  cultural  organization  staif,  and  arts  researchers  about:  (1)  current  practices  in  assessing 
student  learning  in  the  arts;  (2)  the  types  of  assessments  being  using  to  measure  student  learning;  (3) 
how  assessment  tools  are  identified/developed  and  used  in  the  field;  and  (4)  what  assistance  the  field 
needs  to  improve  assessment  of  student  learning  in  the  arts. 

This  report  begins  with  a brief  history  of  assessment  in  the  arts,  including  overviews  at  both  the 
national  and  state  levels.  Chapter  2 presents  the  study  methodology  with  detailed  descriptions  of  the 
data  collection  methods  used  for  the  study.  Study  findings  are  separated  into  two  chapters  — 

Chapter  3 presents  findings  from  the  literature  review  and  Chapter  4 presents  findings  from  the 
survey.  The  final  section  of  the  report  draws  conclusions  about  the  current  status  of  arts  assessment 
and  includes  recommendations  for  moving  the  field  forward. 


THE  HISTORY  OF  ASSESSMENT  IN  THE  ARTS 


NATIONAL  ASSESSMENT  OF  THE  ARTS 

A major  impetus  for  assessment  of  the  arts  at  the  national  level  came  in  1994  when  the  Goals  2000: 
Educate  America  Act  was  signed  into  law.  It  stated  that  by  the  year  2000,  students  in  grades  4,  8,  and 
12  would  have  to  demonstrate  competency  in  a number  of  subject  areas,  including  the  arts  [Public 
Law  103-227,  Section  102(3) (A)].  However,  in  order  to  demonstrate  competency,  standards  first 
needed  to  be  formulated.  Thus,  also  in  1994,  the  National  Standards  for  Arts  Education  were 
developed  (Consortium  of  National  Arts  Education  Association,  1994).  These  were  a set  of 
voluntary  content  and  achievement  standards  for  K— 12  students  studying  dance,  music,  theater,  and 
visual  arts  specifying  what  school  children  should  know  and  be  able  to  do  in  these  disciplines.  In 
addition  to  the  development  of  the  National  Standards  for  Arts  Education,  the  Arts  Education 
Consensus  Project,  sponsored  by  the  National  Assessment  Governing  Board  (NAGB),  began  to 
establish  both  objectives  for  arts  instruction  and  an  assessment  framework  for  the  art  forms  for  the 
fourth,  eighth,  and  twelfth  grades  (CCSSO,  1994).  This  parallel  development  of  two  overlapping 
sets  of  art  standards  was  cited  by  several  expert  consultants  as  a major  event  in  the  history  of  arts 
assessment. 
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In  1997,  the  National  Assessment  of  Educational  Progress  (NAEP),  which  has  its  test  specifications 
set  by  the  NAGB,  began  its  long-term  trend  assessment  of  the  arts  based  on  the  work  by  the  Arts 
Education  Consensus  Project.  The  assessment  included  music,  theater,  and  visual  arts.  Since  1969, 
the  congressionally  mandated  NAEP  was  the  only  nationally  representative  and  continuing 
assessment  of  what  America’s  students  know  and  can  do  in  various  subjects,  including  English- 
language  arts,  mathematics,  science,  history,  and  geometry  to  name  a few  (I'nited  States  Department 
of  Education  [USDE],  n.d.,  a).  The  NAEP  has  assessed  student  knowledge  in  some  form  of  the 
fine  arts  on  multiple  occasions.  Eor  example,  NAEP  conducted  assessments  in  music  in  1971,  visual 
arts  in  1975,  and  both  in  1978  (Oliver,  2007).  However,  these  previous  efforts  were  developed  and 
administered  without  a set  of  uniform  content  standards.  In  contrast,  the  1997  NAEP  Arts 
Assessment  was  based  on  an  arts  framework  that  described  the  specific  knowledge  and  skills  that 
should  be  assessed  in  the  arts  disciplines  (CCSSO,  1994).  In  fact,  the  framework  for  the  NAEP 
envisioned  the  arts  having  an  important  place  in  education: 

The  entire  NAEP  arts  consensus  framework  process  is  founded  on  a vision  of  a society 
that  believes  the  arts  are  essential  to  every7  child’s  complete  development.  Throughout 
their  lives,  they  will  draw  activities,  experience,  and  knowledge  as  a means  of 
understanding  what  happens  both  inside  and  outside  their  own  skin,  just  as  they  use 
mathematical,  scientific,  historical,  and  other  frameworks  for  understanding  (NAGB, 

1997,  p 1). 

For  the  1997  NAEP  Arts  Assessment,  a nationally  representative  sample  of  eighth  grade  students 
was  selected  for  the  music  and  visual  arts  assessments.  Those  who  participated  in  the  theater 
assessment  were  selected  from  a special  targeted  sample  from  schools  that  offered  theater  courses 
and  from  eighth  grade  students  who  took  those  courses  (NAGB,  1997). 2 The  assessment  revolved 
around  three  “art  processes.”  The  first  process  was  creating,  which  referred  to  the  expression  of 
ideas  and  feelings  in  the  form  of  an  original  art  work.  The  second  process,  perfonnance,  referred  to 
performing  an  existing  work  and  applied  only  to  the  performing  arts.  Both  creating  and 
performance  were  assessed  through  a series  of  performance  tasks.  Respondingwas  the  third  process 
and  referred  to  observing,  describing,  analyzing,  and  evaluating  works  of  art,  and  was  assessed 
through  open-ended  and  multiple-choice  questions. 

Because  the  NAEP  Arts  Assessment  is  an  assessment  of  long-term  trends,  it  is  designed  to  be 
administered  less  frequently  than  assessments  of  other  subject  areas  such  as  reading  and 
mathematics.  Therefore,  the  next  NAEP  Arts  Assessment  was  not  administered  until  2008.  This 
time,  however,  it  had  to  be  scaled  back  considerably  due  to  budgetarv  constraints.  For  example, 
only  the  responding  process  in  music,  and  both  the  responding  and  creating  processes  in  visual  arts 
were  assessed.  In  addition,  there  was  no  assessment  of  theater  in  the  2008  NAEP.  Finally,  because 


9 . . 

z Originally,  the  framework  for  the  1997  NAEP  called  for  an  assessment  of  dance  skills;  however,  there  were  only  a 
small  number  of  dance  programs  in  schools;  therefore,  a teacher  survey  about  instructional  practices  in  dance  was 
administered  instead  (Yan  & Reider,  2001). 
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the  scoring  procedures  for  most  of  the  1997  NAEP  Arts  Assessments  could  not  be  replicated  for 
the  2008  NAEP  Arts  Assessment,  comparisons  could  be  made  only  between  students’  performance 
on  the  multiple-choice  questions  in  those  two  years  (Keiper,  Sandene,  Persky,  & Kuang,  2009). 
There  are  plans  to  once  again  administer  the  long-term  trends  NAEP  assessment  to  eighth-grade 
students  in  the  areas  of  music  and  visual  arts  in  2016  (USDE,  n.d.,  b). 


STATE-LEVEL  ASSESSMENT  OF  THE  ARTS 

Soon  alter  their  formulation  in  1994,  the  National  Standards  for  Arts  Education  began  to  serve  as 
the  model  for  states  as  they  developed  their  own  arts  standards.  The  work  of  two  groups  had  a 
profound  effect  on  state-level  assessments  — the  Arts  Education  Partnership  (AEP),3  which  released 
a guide  for  states  and  localities  to  help  them  develop  assessments  in  the  arts,  and  the  State 
Collaborative  on  Assessment  and  Student  Standards,  Arts  Education  Assessment  Consortium 
(SCASS/Arts),  which  was  developed  through  the  Council  of  Chief  State  School  Officers  (CCSSO). 
The  SCASS  has  offered  opportunities  for  states  to  work  together  to  develop  assessments  related  to 
content-based  standards.  Specifically,  the  SCASS/Arts  Consortium  developed  a number  of  training 
materials  for  professional  development  and  a collection  of  performance  exercises  and  assessment 
items.  The  items  are  all  in  a searchable  database,  available  to  members  for  use  in  constructing  arts 
assessments.  One  expert  consultant  commented  on  the  influence  of  SCASS/Arts: 

The  development  of  the  SCASS/Arts  group  was  important  for  bringing  together  leaders 
and  future  leaders  in  arts  assessment.  From  that  group,  we  saw  and  discussed  the 
possibilities  for  arts  assessment.  We  had  deep  conversations  about  what  it  should  be  and 
talked  about  models. . . 

According  to  several  expert  consultants,  a national  conference  on  arts  assessments  held  in  the  1990s 
in  Hidden  Valley,  PA  also  was  a seminal  event.  The  conference  invited  leaders  in  the  arts  from  a 
number  of  states  to  convene  to  discuss  arts  assessment.  After  this  event,  the  arts  assessment  work 
of  SCASS/Arts  and  the  states  gained  momentum.  One  expert  consultant  noted: 

(The  Hidden  Valley  conference]  really  reshaped  our  expectations  and  thinking  about 
assessment.  The  conference  helped  us  see  that  assessment  was  not  just  summative  for 
grading  students,  but  that  it  could  also  be  used  to  help  students  improve  and  to  help 
them  better  understand  the  expectations. 

Adoption  of  art  standards  by  the  states  occurred,  for  the  most  part,  in  the  mid-  to  late- 1990s  (Yan  & 
Reider,  2001).  A notable  exception  is  Illinois,  which  first  developed  art  education  standards  in  1985. 
State-level  assessments  in  the  fine  arts  began  in  1993,  with  Kentucky  being  the  first  state  to  make 


3 The  Arts  Education  Partnership  is  a private,  nonprofit  coalition  of  more  than  100  national  education,  arts,  business, 
and  philanthropic  organizations  that  was  formed  in  1995  through  a cooperative  agreement  between  the  National 
Endowment  for  the  Arts,  the  U.S.  Department  of  Education,  the  National  Assembly  of  State  Arts  Agencies  (NASAA), 
and  the  Council  of  Chief  State  School  Officers  (CCSSO).  It  is  administered  by  CCSSO. 
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such  an  art  education  assessment  mandatory.  The  next  state  to  adopt  an  arts  assessment  was  Alaska 
in  1995,  although  in  this  case  the  assessment  was  voluntary.  By  2001,  approximately  one-third  of 
states  had  either  a mandatory  or  voluntary  statewide  arts  assessment.  Specifically,  six  states  (Illinois, 
Kentucky,  Maine,  Minnesota,  New  Jersey,  Oklahoma)  mandated  assessment  of  fine  arts,  three  states 
(Missouri,  New  York,  Washington)  mandated  assessments  that  were  to  begin  in  the  next  several 
years,  and  seven  states  (Alaska,  California,  Massachusetts,  Ohio,  Pennsylvania,  Utah,  and  Vermont) 
reported  a system  of  voluntary'  arts  assessments.  Typically,  students  were  assessed  in  4'  or  S'  grade, 
6,h  or  7'1’  grade,  and  10lh  or  1 1th  grade  in  dance,  music,  theater,  and/or  visual  arts  using  a combination 
of  multiple-choice  and  constructed-response  items.  At  the  time,  the  arts  assessments  were 
considered  high-stakes  in  only  two  states  - Kentucky  used  the  results  for  school  accountability 
purposes  and  Minnesota  used  individual  student  results  as  part  of  a high  school  exit  exam  for 
graduation. 

The  proportion  of  states  requiring  assessments  in  fine  arts  increased  over  time.  According  to  the 
most  recent  survey  of  state  education  agencies  conducted  by  the  Arts  Education  Partnership  (AEP) 
during  the  2007-2008  school  year,  47  states  and  the  District  of  Columbia  reported  they  had  arts 
education  state  standards  and  13  states  reported  a state-mandated  assessment  in  the  fine  arts.  At  the 
time,  only  one  of  these  states,  Kentucky,  required  that  a state-level  assessment  instrument  be  used 
(AEP,  n.d.).  Kentucky  has  since  removed  the  arts  assessment  portion  from  the  state  core  content 
assessment  and,  as  such,  no  longer  requires  arts  competency  to  be  demonstrated  through  a state- 
level  assessment  (Kentucky  Department  of  Education,  2009).  The  other  12  states  required  either 
the  district  or  school  to  develop  their  own  assessment  or  to  use  any  exisdng,  validated  instrument. 

In  addition,  several  other  states  have  voluntary  arts  assessment  (AEP,  n.d.). 

Given  the  increased  pressure  for  assessment  and  accountability'  since  the  1990’s,  the  NEA  identified 
the  need  to  capture  the  current  status  of  arts  assessment.  In  2005,  the  NEA  began  requiring  a 
narrative  statement  of  assessment  practices  to  apply  for  arts  education  funds.  Project  applicants 
needed  to  explain  their  assessments  methods  and  types  of  tools  used  to  measure  student  knowledge 
and  skills.  Through  several  grant  cycles,  it  became  clear  to  NEA  staff  that  applicants  did  not 
necessarily  differentiate  between  program  evaluation  and  assessment  of  student  learning.  As  such, 
the  NEA  commissioned  this  study  to  examine  current  trends,  promising  techniques,  and  successful 
practices  being  used  to  assess  student  learning  in  the  arts  throughout  the  country7.  Although  the 
original  intent  of  the  study  was  to  identify  strong  models  of  assessment  practices  that  could  serve  as 
examples  for  possible  replication,  the  study  found  that  such  models  were  not  available  and  are  in 
fact  a need  of  the  field.  Thus,  this  report  provides  a description  of  the  current  state  of  arts 
assessment,  including  a review  of  the  high-quality  literature  available,  common  practices  being  used 
to  assess  student  learning,  and  needs  of  the  field  to  improve  arts  assessment.  The  following  chapters 
present  the  methods  used  and  findings  that  resulted  from  the  study. 
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Chapter  2: 
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Study  Methodology 

The  study  was  designed  to  capture  the  current  status,  trends,  and  practices  in  assessing  student 
learning  in  the  arts.  The  first  step  was  to  clearly  define  student  learning  in  the  arts  and  student 
assessment  in  the  arts  in  order  to  keep  the  study  focused  and  clear.  The  study  consisted  of  three 
major  methodological  components:  (1)  consultation  with  experts  in  the  field;  (2)  a literature  review 
of  information  about  practices  in  assessing  student  learning  in  the  arts;  and  (3)  a survey  of  members 
and  grantees  from  national  arts  organizations. 

WestEd  collaborated  with  the  NEA  to  identify  key  leaders  in  the  field  of  assessment  of  student 
learning  in  the  arts,  and  WestEd  researchers  consulted  with  21  key  leaders  throughout  the  country. 
These  expert  consultants  shared  their  perspectives  on  current  assessment  trends  and  practices, 
identified  additional  sources  of  literature  for  review,  and  provided  input  into  and  reviewed  the 
project  survey. 

The  literature  review  component  was  designed  to  collect  information  about  assessing  student 
learning  in  the  arts,  such  as  assessment  tools,  resources  on  assessment,  informational  websites, 
technical  research  and  evaluation  reports,  and  other  accessible  documents  and  materials.  The 
purpose  was  to  determine  the  current  state  of  what  was  available  to  the  field  and  identify  strengths 
and  gaps  that  may  exist.  For  each  type  of  literature  reviewed,  a summary  of  the  high-quality 
materials  and  examples  of  both  high-  and  low-quality  documents  are  presented. 

The  survey  was  designed  to  collect  information  directly  from  individual  members  and  grantees  of 
national  arts  organizations  about  their  experiences  assessing  student  learning  in  the  arts;  the  types  of 
assessments  used  to  measure  student  learning  in  the  arts;  and  what  assistance  they  feel  is  needed  to 
improve  the  assessment  of  student  learning  in  the  arts. 

The  following  sections  provide  working  definitions  and  assumptions  related  to  the  study,  followed 
by  detailed  descriptions  of  the  data  collection  methods  used. 


DEFINING  STUDENT  LEARNING  IN  THE  ARTS 


Student  learning  in  the  arts  can  be  viewed  in  multiple  ways.  For  the  purposes  of  this  study,  student 
learning  in  the  arts  was  defined  as  acquiring  knowledge  and/or  skills  in  one  or  more  art  forms. 
Specifically,  art  knowledge  refers  to  student  content  knowledge  such  as  history,  terminology, 
recognition,  and  cultural  relevance.  An  example  of  art  knowledge  is  students’  ability7  to  identify  a 
particular  dance  step,  whether  or  not  they  are  capable  of  performing  it.  In  contrast,  art  skills  refers 
to  students’  actual  abilities  to  perform  or  produce  art,  such  as  how  well  students  perform  the  dance 
step. 

For  the  purpose  of  this  study,  the  definition  of  student  learning  in  the  arts  was  limited  to  art-specific 
learning.  The  study  did  not  examine  and  does  not  include  other  areas  that  can  be  impacted  by  arts 
participation  such  as  outcomes  in  the  cognitive  (e.g.,  academic  achievement  in  other  core  subjects) 
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and  affective  (e.g.,  self-confidence,  attitudes)  domains.  As  such,  reports  that  presented  findings 
from  through-the-arts  studies  (i.e.,  using  the  arts  as  an  entry  point  to  improve  other  academic  areas) 
were  excluded,  as  were  studies  that  focused  on  areas  other  than  arts  knowledge  and  skills,  such  as 
creativity,  arts  appreciation,  and  participation  in  arts-related  events. 

The  definition  of  student  was  limited  to  those  currently  enrolled  in  kindergarten  through  twelfth 
grade,  so  studies,  resources,  and  tools  focused  at  the  university  or  adult-learner  levels  were  excluded 
from  the  study.  Further,  to  be  included  in  the  study  analysis,  the  information  needed  to  be 
produced  in  or  include  information  addressing  the  United  States.  More  specifically,  studies  limited 
to  students  irom  other  countries  were  excluded,  but  resources  or  assessment  tools  produced  in 
another  country  but  currently  available  and  in  use  in  the  United  States  were  included. 


DEFINING  ASSESSMENT  OF  STUDENT  LEARNING  IN  THE  ARTS 


For  the  purposes  of  this  study,  the  assessment  of  student  learning  in  the  arts  was  defined  as  the 
measurement  of  students’  knowledge  and/or  skills  in  one  or  more  art  forms.  In  order  to  be 
considered  for  the  study,  an  assessment  tool  needed  to  be  designed  to  measure  some  type  of  art 
knowledge  or  art  skill  (i.e.,  in  the  arts).  Assessments  designed  to  measure  other  outcomes,  such  as  the 
affective  and  cognitive  domains  (i.e.,  through  the  arts)  were  not  included  as  a part  of  this  study.  Art 
forms  consisted  of  dance,  folk  arts,  literary  arts,  media  arts,  music,  musical  theater,  opera,  theater, 
and  visual  arts. 

All  modes  of  assessment  designed  to  measure  student  learning  in  the  arts  were  incorporated  into  the 
study.  Examples  of  the  types  of  knowledge  and  skills  assessments  included  computer-based 
software,  paper-pencil  assessments,  observation  protocols,  performance-based  assessments, 
portfolio  reviews,  checklists,  rubrics,  student  self-assessments,  and  teacher/artist  surveys  (see  Page 
95  - Definition  of  Terms  for  a description  of  each  type  of  assessment  tool).  In  order  to  provide 
needed  context  for  understanding  the  current  status  of  arts  assessment  in  the  United  States,  the 
study  also  examined  the  origin  of  assessments,  why  they  were  developed,  and  how  data  were  used. 


DATA  COLLECTION  METHODS 


EXPERT  CONSULTATION 

WestEd  collaborated  with  the  NEA  to  identify  key  leaders  in  assessment  of  student  learning  in  the 
arts  who  could  share  their  perspectives  on  current  assessment  trends  and  practices  as  well  as 
indentifv  additional  sources  of  literature  for  review.  WestEd  consulted  with  21  leaders  from 
agencies  across  the  countrv,  including  the  United  States  Department  of  Education  (USDE),  national 
arts  organizations,  state  education  agencies,  state  arts  councils,  foundations,  regional  educational 
laboratories,  and  large  school  districts. 
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These  expert  consultants  were  invaluable  to  the  study  in  a number  of  ways,  such  as  engaging  in  in- 
depth  conversations  about  arts  assessment,  identifying  key  information,  reviewing  studv  methods 
and  tools,  making  recommendations,  and  in  some  cases,  assisting  with  the  administration  of  the 
survey.  The  expert  consultants  provided  information  related  to  the  history  of  arts  education  and 
assessment,  how  assessment  was  used  and  what  was  learned  from  it,  current  expectations  of  the 
assessment  of  student  learning,  definitions  of  art  knowledge  and  skills,  views  of  assessment  and  the 
needs  of  the  field,  issues  and  challenges  of  assessment,  the  various  types  of  assessments  used  in  the 
arts,  policy  issues,  budget  issues,  and  issues  related  to  arts  access  and  equity.  Their  input  helped 
guide  survey  development  and  provided  valuable  information  that  cut  across  different  areas  of 
experdse,  geography,  demographics,  art  forms,  and  levels  of  involvement  with  arts  education  and 
arts  assessment. 

All  of  the  conversations  with  the  expert  consultants  took  place  over  the  phone  between  January  and 
February  2009.  Some  of  the  expert  consultants  also  reviewed  the  survey  instrument  and  provided 
feedback  before  the  survey  was  administered. 


LITERATURE  REVIEW  METHODS  AND  PROCESS 

For  the  purposes  of  this  study,  literature  was  a broad  term  used  to  describe  documents,  websites, 
journal  articles,  assessment  tools,  curricula,  resource  guides,  research  and  evaluation  reports,  and 
other  informational  materials  that  could  provide  insight  into  the  current  status  of  assessment  of 
student  learning  in  the  arts.  The  review  covered  both  recentlv  developed  and  older  materials, 
including  some  assessment  tools  developed  in  the  1960s.  Prior  to  beginning  the  literature  review 
process,  the  study  team  identified  a list  of  keywords  that  guided  the  search.  The  words  were 
grouped  into  topics  including: 

• Art  forms  — dance,  drama,  literary  arts,  media  arts,  music,  musical  theater,  theater 
(and  alternate  spelling  theatre),  creative  writing,  poetry,  photography,  and  visual  arts 

• Types  of  documents  — assessment  tool,  resource  guide,  research  report,  program 
evaluation,  item  bank 

• Types  of  assessment  tools  — measure,  test,  protocol,  instrument,  rubric,  portfolio 

The  study  team  then  entered  individual  and  groups  of  words  into  various  ty  pes  of  search  engines, 
including  Internet  search  engines  (e.g.,  Google,  Yahoo,  HotBot,  Ask),  journal  article  search  engines 
(e.g.,  Education  Resources  Information  Center  [ERIC],  ProQuest  Journal  Database),  and  test  bank 
search  engines  (e.g.,  Buros  Institute  of  Mental  Measurement).  How  kev  words  and  groups  were 
entered  depended  on  the  search  engine.  For  example,  when  searching  the  Buros  test  database, 
single  words  such  as  “music”  or  “dance”  were  used  because  the  database  only  contains  assessment 
tools.  When  searching  using  an  Internet  engine  such  as  Google  or  Yahoo,  multiple  words  were  used 
to  narrow  the  search,  such  as  “dance  knowledge  assessment”  or  “visual  arts  evaluation  report.” 
Combinations  of  words  were  used  with  each  art  form  (e.g.,  “dance  skills  rubric,”  “music  skills 
rubric,”  “theater  skills  rubric”).  When  searching  a journal  database,  such  as  ProQuest  or  ERIC,  a 
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wide  net  was  cast  and  then  narrowed  depending  on  the  number  of  articles  found.  For  example,  the 
search  might  start  with  just  the  word  “music”  and  the  first  50-100  articles  were  reviewed  to  see  what 
types  of  articles  were  found.  Then  the  search  was  narrowed  by  adding  or  excluding  words  from  the 
search.  When  searching  for  “visual  arts,”  the  words  “liberal”  and  “English  language”  were  excluded 
trom  the  search  because  of  the  large  number  of  irrelevant  sites  that  were  identified  by  the  search 
engines.  In  addition  to  search  engines,  specific  arts-related  websites  (e.g.,  NEA,  Arts  Education 
Partnership,  Americans  for  the  Arts,  Harvard  Project  Zero)  were  searched  to  identify  potentially 
relevant  documents. 

For  each  search,  a WestEd  staff  member  would  review  the  websites  identified  by  the  search  engine 
to  determine  if  the  site  or  article  contained  information  or  resources  relevant  to  the  study.  Irrelevant 
sites  were  ignored,  such  as  those  pertaining  to  other  countries  or  older  populations  (e.g.,  university 
students).  Websites,  resources,  research,  and  evaluation  reports  that  focused  on  learning  through  the 
arts  rather  than  learning  in  the  arts  (e.g.,  focused  on  improving  math  by  teaching  music  rather  than 
focused  on  improving  music  skills)  were  also  excluded  trom  the  current  study.  Literature  identified 
as  having  some  potential  relevance  to  the  study  was  entered  into  a Master  Tracking  database  for 
further  review. 

Each  document  entered  into  the  Master  Tracking  database  was  assigned  an  item  code  that  identified 
the  art  form  addressed,  the  type  of  document,  and  an  individual  item  number.  The  art  forms 
included  were  cross-disciplinary,  dance,  general  information  (not  specific  to  the  arts  but  could  be 
used  with  the  arts),  literary'  arts,  media  arts,  music,  theater,  and  visual  arts.  The  types  of  documents 
were  separated  into  the  following  categories:4 

Assessments  — Assessments  were  defined  as  tools  or  tests  specifically  designed  for 
measuring  knowledge  or  skills  in  the  arts.  This  included  paper-pencil  tests,  online  tests, 
rubrics,  portfolio  reviews,  observation  protocols,  and  checklists.  Examples  of 
assessments  included  the  Advanced  Placement  tests  in  the  arts,  teacher-developed 
rubrics,  and  state  education  agency  tests. 

Collections  - Collections  were  defined  as  groups  of  assessments  or  lesson  plans  with 
assessments.  This  included  sets  of  tools  within  an  art  form  that  provided  four  or  more 
separate  assessments,  such  as  a group  of  assessments  that  covered  different  topics  or 
different  grade  levels.  Examples  of  collections  included  school  district  or  arts 
organization  websites  that  allowed  members  to  upload  lessons  and  assessments, 
textbook  publisher  books/curricula  that  included  lessons  and  assessments  for  multiple 
grade  levels,  and  articles  that  presented  multiple  assessment  tools  within  an  art  form 
(e.g.,  one  document  or  website  containing  separate  assessments  for  piano,  guitar,  violin, 
drums,  trumpet,  and  flute). 


4 Some  literature  presented  information  that  could  pertain  to  multiple  document  types.  When  this  occurred,  the  study 
team  reviewed  the  materials  and  identified  the  most  appropriate  fit  to  keep  the  categories  mutually  exclusive  and  not 
over-represent  a single  type  of  document.  For  example,  nearly  all  resources  would  also  qualify  as  informational 
documents,  but  informational  materials  needed  to  provide  how-to  instruction  to  qualify  as  resource  documents. 
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Informational  — Informational  documents  were  defined  as  those  that  provided 
educational  information  about  assessing  knowledge  and  skills  in  the  arts.  This  included 
overviews,  definitions  of  terms,  policies,  standards,  and  detailed  descriptions  of  student 
assessment  in  the  arts.  Examples  of  informational  documents  included  state  frameworks 
for  arts  education,  books  that  presented  different  types  of  assessments  and  how  they  are 
used,  articles  discussing  theoretical  approaches  to  assessment  of  arts  education,  and 
websites  that  complied  links  to  resources  from  other  agencies  or  organizations. 

Technical  Reports  — Technical  reports  were  defined  as  research  and  evaluation  reports 
that  included  the  assessment  of  student  knowledge  and/or  skills  as  an  outcome  variable. 

In  cases  where  multiple  annual  reports  were  produced,  only  the  most  recent  report  was 
reviewed.  An  example  of  a literature  report  would  be  the  final  evaluation  report  that  an 
arts  organization  and/or  external  evaluator  developed  as  a funding  requirement. 

Resources  — Resources  were  defined  as  “how-to”  guides.  These  documents  provided 
readers  with  instructions  on  how  to  develop  assessments  or  how  to 
implement/administer  assessments.  An  example  of  a resource  would  be  a step-by-step 
guide  on  how  to  develop  a rubric  to  assess  student  skills  in  the  arts. 

For  each  type  of  document,  a separate  categorical  database  was  developed  to  catalogue  and  track 
information  relevant  to  that  type  of  document.  For  example,  the  assessment  database  included 
categories  for  age  range;  whether  the  tool  assessed  knowledge,  skills,  or  both;  whether  validity  and 
reliability'  information  were  available;  and  the  extent  of  the  use  (e.g.,  national,  statewide,  local).  In 
contrast,  categories  in  the  informational  database  included  the  intended  audience  and  the  topics 
addressed,  such  as  classroom  assessment,  program  evaluation,  or  defining  goals  and  objectives. 

Each  piece  of  literature  entered  into  the  Master  Tracking  database  underwent  a two-step  review 
process.  The  initial  review  step  was  to  gauge  the  relevance  of  the  information  to  the  study  (i.e., 
focused  on  assessment  of  K-12  student  knowledge  and/or  skills  in  the  arts).  As  items  were  entered 
into  the  Master  Tracking  database,  they  were  assigned  a relevance  rating.  Relevance  ratings  were 
based  on  a 10-point  scale  where  1 = little  to  no  relevance  to  the  current  study  (e.g.,  was  off-topic  or 
did  not  actually  address  student  knowledge  or  skills  in  the  arts)  and  10  = extremely  relevant,  such  as 
an  assessment  tool  that  directly  measured  student  knowledge  in  the  arts  or  a resource  that  provided 
a step-by-step  guide  on  how  to  develop  an  arts-assessment  rubric.  Items  were  also  coded  as  to 
whether  they  were  worthy  of  a secondary7  review  or  if  the  initial  review  was  sufficient.  For  example, 
if  the  initial  review  determined  a study  was  about  how  music  instruction  improved  academic 
achievement,  it  would  receive  a low  rating  for  relevance  and  the  second  step  in  the  review  process 
was  not  necessary7  since  the  article  did  not  measure  students’  musical  knowledge  or  skills.  In  general, 
items  that  received  a relevance  rating  lower  than  5 were  excluded  from  further  review.  Of  the  727 
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items  included  in  the  Master  Tracking  database,  395  (53.9%)  were  excluded  from  further  analysis 
because  they  did  not  meet  the  minimum  threshold  for  relevance.  5 

The  second  step  of  the  review  process  was  to  gauge  the  quality  of  the  literature  that  was  relevant 
enough  for  inclusion  in  the  study.  Quality  ratings  ranged  from  1 (extremely  low  quality)  to  10 
(extremely  high  quality),  with  1-3  being  considered  lower  quality',  4-6  being  considered  moderate 
quality,  and  7-10  being  considered  higher  quality.  Quality  was  gauged  entirely  in  terms  of  the 
content  ol  the  information,  without  rating  for  aesthetics  or  other  less-relevant  factors.  Quality'  of 
content  was  based  on  the  information  provided  for  the  tvpe  of  document  and  the  appropriateness  to 
the  intended  audience.  For  example,  a higher  quality  assessment  tool  would  have  reliability  and 
validity  information  available,  while  a higher  quality  resource  would  have  clear  and  easy  to  follow 
instructions.  The  guidelines  used  to  gauge  quality  for  each  type  of  document  are  presented  in 
Appendix  A. 

Documents  needed  to  receive  a minimum  quality  rating  of  7 to  be  entered  into  the  categorical 
databases.  The  two-step  review  process  helped  ensure  that  only  relevant  materials  with  an  adequate 
quality  level  were  included  in  the  literature  analysis.  Of  the  297  documents  that  met  the  criteria  for 
relevance,  148  (49.8%)  received  a high  enough  quality'  rating  (i.e.,  rated  7 or  above)  to  be  included  in 
the  categorical  databases.  Finalized  categorical  databases  were  then  reviewed  and  summarized  for 
the  report,  and  findings  are  presented  in  Chapter  3. 


NATIONWIDE  SURVEY 

To  capture  a more  thorough  picture  of  the  current  status  of  arts  assessment,  the  survey  was 
designed  to  collect  information  directly  from  school  and  district  staff,  arts  and  cultural  organizadon 
staff,  state  and  county  policy  makers,  researchers,  and  evaluators  about:  (1)  how  and  why 
respondents  assess  student  learning  in  the  arts;  (2)  the  types  of  arts  assessments  used  to  measure 
student  arts  learning;  (3)  the  experience  respondents  have  assessing  students’  knowledge  and  skills  in 
the  arts;  and  (4)  what  assistance  is  needed  to  improve  assessment  of  student  learning  in  the  arts. 

Another  aspect  of  the  survey  was  to  collect  additional,  non-published  materials  developed  by  or  for 
the  respondents,  such  as  assessment  tools  and  research/evaluation  reports.  Teachers  and  teaching 
artists  often  develop  their  own  assessment  tools  to  measure  student  progress,  and  arts  organizations 
conduct  research  or  evaluations  that  do  not  necessarily  culminate  in  a publication  or  publicly 
distributed  report.  For  example,  arts  organizations  may  conduct  an  evaluation  to  fulfill  a grant 
requirement  and  the  report  never  goes  beyond  the  funder.  This  information  is  nonetheless 
important  to  understanding  the  current  state  of  arts  assessment  and  survey  respondents  were  able  to 


^ For  38  of  the  identified  documents,  the  study  team  was  unable  to  obtain  an  actual  copy  of  the  item  or  sufficient 
information  about  the  item  from  other  sources  to  conduct  an  adequate  review.  Reasons  for  not  obtaining  the  items 
included  them  being  out  of  print,  requests  for  review  copies  were  not  returned,  websites  initially  identified  were  later 
disabled,  and  the  cost  of  the  item  made  it  unfeasible  to  obtain.  As  such,  the  overall  relevance  and  quality  of  these 
materials  could  not  be  determined  and  they  were  not  included  in  the  study  analysis. 
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share  unpublished  reports  and  materials  via  a special  email  address  created  for  this  study.  The 
materials  received  were  included  as  part  of  the  literature  review  process,  entered  into  the  Master 
Tracking  database,  rated  for  relevance  and  quality,  and  entered  into  the  categorical  databases  as 
appropriate.  A total  of  71  items  were  received  through  the  study  email  address  and  8 (1 1.3%)  met 
the  standards  for  quality  and  relevance  for  inclusion  in  the  study. 

To  maximize  the  value  and  efficiency  of  the  survey,  it  was  developed  concurrent  with  the  initial 
review  of  the  literature  and  after  consultation  with  experts  in  the  field.  Thus,  questions  reflected 
what  was  being  learned  from  the  preliminary  literature  review  and  expert  consultants,  particularly  as 
it  related  to  state  and  national  trends  in  assessment  of  student  learning  in  the  arts.  This  allowed  for 
more  targeted  questions  that  probed  deeper  to  collect  richer  data  on  trends  and  practices.  Further, 
the  survey  served  as  a formative  tool  to  help  identify  the  needs  of  the  field  anti  formulate 
recommendations  on  ways  to  help  improve  the  state  of  assessment  of  K-12  student  learning  in  the 
arts. 


SURVEY  PROCEDURES 

WestEd  worked  closely  with  NEA  staff  throughout  the  design  and  revision  process,  and  enlisted  key 
leaders  in  the  area  of  arts  assessment  to  review  the  survey  as  well. 

WestEd  pilot-tested  the  survey  in  two  phases.  The  first  pilot,  prior  to  OMB  approval,  included  a 
group  of  four  respondents  to  assess  item  comprehension,  the  effectiveness  of  the  proposed 
strategies  for  gaining  participation,  and  the  length  of  time  needed  for  respondents  to  complete  the 
survey  questions.  Such  information  helped  determine  the  actual  burden  associated  with  the  survey 
(e.g.,  30  minutes  per  respondent).  The  second  pilot  test  occurred  after  OMB  approval.  The 
approved  survey  was  pilot-tested  with  members  of  one  of  the  national  arts  organizations  that  agreed 
to  send  the  survey  to  its  members.  This  pilot  helped  ensure  data  were  received  in  the  intended 
format,  allowed  for  preliminary  coding  and  categorization  of  qualitative  data,  and  tested  procedural 
methods  before  continuing  the  use  of  the  survey. 

The  intention  of  the  study  was  to  capture  the  perspectives  of  a wide  variety  of  arts  educators  (e.g., 
teachers,  teaching  artists,  arts  specialists),  policymakers,  and  researchers/evaluators  who  represented 
many  art  forms,  age  ranges,  instructional  settings,  organizations,  and  populations.  Thus,  WestEd 
administered  the  survey  in  cooperation  with  national  arts  organizations,  education  agencies,  cultural 
organizations,  and  research  organizations  (referred  to  as  partner  agencies).  Respondents  were 
grantees  or  members  of  the  listserv  or  email  list  of  the  partner  agencies.  This  pool  of  individuals 
provided  rich  data  and  described  the  processes  in  which  they  and  their  agency/ organization  have 
used  arts  assessment  tools.  Although  there  are  limitations  in  using  such  a sampling  method,  the  goal 
of  the  study  was  to  better  understand  the  current  status  of  arts  assessment,  which  did  not  necessitate 
specialized  sampling  techniques  or  precision  in  defining  the  universe  or  potential  respondents. 

Fifteen  partner  agencies  agreed  to  collaborate  with  WestEd  to  administer  the  survey  to  their 
members,  grantees,  and/or  listserv  group.  The  partner  agencies  were: 
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1.  American  Alliance  for  Theater  and  Education  (AATE) 

2.  American  Evaluation  Association  (AEA)  Arts  and  Culture  Topical  Interest  Group  (TIG) 

3.  Council  for  Chief  State  School  Officers  (CCSSO)/Arts  Education  Partnership  (AEP) 

4.  Cultural  Arts  Resources  for  Teachers  and  Students  (CARTS) 

5.  U.S.  Department  of  Education  (USDE)  - Arts  Education  Model  Development  and 
Dissemination  grantees  and  Professional  Development  for  Arts  Educators  grantees 

6.  Education  Theater  Association  (EdTA) 

7.  National  Guild  of  Community  Schools  of  the  Arts  (GUILD) 

8.  League  of  American  Orchestras  (LEAGUE,) 

9.  National  Alliance  for  Media  Arts  and  Culture  (NAMAC) 

10.  National  Arts  Education  Association  (NAEA) 

1 1.  National  Assembly  of  State  Arts  Agencies  (NASAA) 

12.  National  Dance  Education  Organization  (NDEO) 

13.  National  Endowment  for  the  Arts  (NEA)  — Multiple  arts  education  project  grantees 

14.  MENC  — The  National  Association  for  Music  Education  (MENC) 

15.  State  Education  Agency  Directors  of  Arts  Education  (SEADAE) 

WestEd  worked  directly  with  the  organizations  to  obtain  their  cooperation  in  sending  out  the  survey 
on  behalf  of  the  NEA.6  The  organization  representatives  sent  an  email  to  their  entire  membership, 
and  each  member  (i.e.,  potential  respondent)  was  given  a survey  access  code.  The  email  indicated 
participation  was  voluntary  and  included  the  purpose  for  the  survey,  a link  for  accessing  the  survey, 
and  contact  information  should  respondents  have  any  questions  or  want  additional  information 
about  the  data  collection. 

Participation  in  the  survey  was  voluntary',  and  respondents  were  able  to  opt  out  of  the  study  with  no 
consequence.  Ample  time  was  allocated  for  completing  the  survey.  Additionally,  the  web-based 
survey  enabled  respondents  to  complete  the  survey  in  one  session,  or  in  multiple  sessions  at  times 
convenient  to  them.  The  survey  also  included  skip  patterns  so  that  respondents  would  not  see  or  be 
asked  to  answer  questions  that  did  not  apply  to  them.  A copy  of  the  survey  is  presented  in 
Appendix  B. 

During  the  data  collection  period,  WestEd  also  established  a dedicated  email  address  to  allow 
respondents  to  ask  questions,  report  problems,  or  request  assistance.  The  e-mail  address  was 
featured  on  all  pages  of  the  survey  and  in  any  related  correspondence  to  respondents.  The  email 


6 WestEd  and/or  the  NEA  were  not  permitted  to  send  out  the  survey  link  directly  due  to  listserv  participadon  laws. 
When  individuals  join  a listserv',  the  sponsoring  organization  assures  that  their  email  addresses  will  not  be  shared. 
Therefore,  each  organization  needed  to  send  out  the  survey  to  its  listserv'  members. 
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address  also  allowed  participants  to  share  any  assessment  tools  or  resources  they  developed  for 
inclusion  in  the  literature  review. 

To  maximize  response  rates,  WestEd  worked  with  the  partner  organizations  to  send  out  a pre- 
survey notification  that  provided  advance  notice  of  the  survey  so  potential  respondents  were  aware 
the  survey  would  be  coming  shortly.  During  the  data  collection  process,  WestEd  provided  the 
OMB  approved  text  for  email  reminders,  and  the  partner  agencies  sent  reminder  emails  two  weeks 
after  the  initial  administration,  and  again  one  month  after  the  initial  administration. 

Throughout  the  data  collection  period,  WestEd  tracked  the  response  rates  as  participants  completed 
the  survey.  WestEd  provided  regular  updates  to  the  partner  agencies  and  the  NEA  on  the  response 
rate  and  the  status  of  the  data  collection.  In  cases  where  the  response  rate  was  unexpectedly  low,  we 
worked  with  the  NEA  and  partner  agencies  to  identify  strategies  for  increasing  participation,  such  as 
extending  the  deadline,  sending  out  thank  you  notes,  and  reminding  people  about  the  purpose  of  the 
study. 

RESPONSE  RATES 

For  some  agencies,  their  individual  response  rate  was  quite  high,  such  as  the  National  Assembly  of 
State  Arts  Agencies  (NASAA)  which  had  100%  of  state  arts  agency  Arts  Education  Managers 
complete  the  survey  and  the  U.S.  Department  of  Education  (USDE)  which  had  90.6%  of  its  current 
Arts  Education  Model  Development  and  Dissemination  grantees  complete  the  survey.  I lowever, 
some  partner  agencies’  response  rates  were  extremely  low  due  in  part  to  the  sheer  number  of 
members.  For  example,  the  National  Association  for  Music  Education  (MENC)  has  a listserv 
membership  of  approximately  60,000  people,  but  there  was  no  guarantee  that  all  its  members 
opened  or  read  the  emails  asking  them  to  complete  the  survey.  The  overall  response  rate  across  all 
15  agencies  was  5.1%;  however,  when  MENC  is  removed  the  response  rate  increased  to  27.3%. 
Exhibit  1 shows  the  response  rates  by  partner  agency. 
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Exhibit  1 - Response  Rates  by  Organization 


Partner  Organization 

Number  of 
Respondents 

Number  on 
Distribution  List 

Response  Rate 

NASAA 

44 

44 

100 

USDE 

48 

53 

90.6 

SEADAE 

35 

57 

61.4 

NDEO 

229 

400 

57.3 

NEA 

256 

451 

56.8 

CARTS 

20 

37 

54.1 

GUILD 

232 

433 

53.6 

LEAGUE 

68 

170 

40.0 

NAEA 

1,565 

4,341 

36.1 

AEATIG 

41 

165 

24.8 

EDTA 

857 

4,050 

21.2 

NAMAC 

57 

350 

16.3 

CCSSO/AEP 

178 

2,100 

8.5 

AATE 

11 

700 

1.6 

MENC 

103 

60,000 

0.002 

Total 

3,744 

73,351 

5.1 

DATA  ANALYSIS 

Due  to  survey  administration  through  the  15  partner  agencies,  the  sample  of  participants  was  large 
and  represented  a wide  variety  of  perspectives.  However,  the  sample  was  also  limited  in  that  it  only 
surveyed  members  and  grantees  of  these  partner  agencies.  While  some  agencies  maintain  listservs 
that  are  free  and  open  to  all  people,  other  agencies  require  annual  membership  dues  that  may  limit 
the  reach  to  their  intended  population.  Further,  partner  agencies  that  administered  the  survey  to 
their  grantees  were  only  able  to  survey  staff  from  organizations  that  were  funded  and  not  the  entire 
population  of  organizations  that  applied,  further  Limiting  the  study  sample.  As  such,  the  sample  may 
not  be  representative  of  the  entire  population  of  arts  teachers,  teaching  artists,  policymakers,  and 
researchers.  Despite  these  limitations,  the  survey  respondents  represented  all  50  states,  the  District 
of  Columbia,  and  Puerto  Rico.  As  can  be  seen  in  Exhibit  2,  New  York,  California,  and  Texas  had 
the  greatest  number  of  respondents. 
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Exhibit  2 — States  of  Residence 


Number  of  Respondents 

Percent 

New  York 

308 

8.2 

California 

266 

7.1 

Texas 

205 

5.5 

Florida 

193 

5.2 

Illinois 

172 

4.6 

Pennsylvania 

166 

4.4 

Virginia 

145 

3.9 

Michigan 

131 

3.5 

Ohio 

125 

3.3 

Georgia 

122 

3.3 

Missouri 

120 

3.2 

Maryland 

111 

3.0 

Massachusetts 

109 

2.9 

North  Carolina 

100 

2.7 

Kentucky 

97 

2.6 

New  Jersey 

95 

2.5 

Arizona 

78 

2.1 

Indiana 

78 

2.1 

Minnesota 

75 

2.0 

Wisconsin 

71 

1.9 

Colorado 

68 

1.8 

Washington 

66 

1.8 

South  Carolina 

60 

1.6 

Tennessee 

59 

1.6 

Kansas 

57 

1.5 

Connecticut 

55 

1.5 

Iowa 

50 

1.3 

Oregon 

47 

1.3 

Arkansas 

45 

1.2 

Alabama 

39 

1.0 

New  Mexico 

38 

1.0 

Nebraska 

36 

1.0 

Oklahoma 

30 

0.8 
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Louisiana 

28 

0.7 

Mississippi 

28 

0.7 

Utah 

27 

0.7 

New  Hampshire 

24 

0.6 

Nevada 

21 

0.6 

District  of  Columbia 

20 

0.5 

Vermont 

20 

0.5 

West  Virginia 

18 

0.5 

Alaska 

17 

0.5 

Hawaii 

17 

0.5 

Montana 

17 

0.5 

Rhode  Island 

17 

0.5 

Delaware 

16 

0.4 

Maine 

15 

0.4 

Idaho 

12 

0.3 

South  Dakota 

9 

0.2 

Puerto  Rico 

8 

0.2 

Wyoming 

7 

0.2 

North  Dakota 

6 

0.2 

Total 

3,744 

100 

More  than  hall:  the  survey  respondents  (58.9%)  were  school  or  district  staff,  and  another  20.4%  were 
arts  organization  staff.  Others  who  completed  the  survey  included  university  faculty',  arts 
researchers,  university'  students  currently  enrolled  in  arts  and  education  programs,  state  and  county' 
staff,  arts  evaluators,  and  freelance  artists  (Exhibit  3). 
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Exhibit  3 - 

Primary  Role  of  Survey  Respondents 

Number  of  Respondents 

Percent 

School/District  Staff 

2,207 

58.9 

Arts  Organization  Staff 

764 

20.4 

University  Faculty/Staff 

187 

5.0 

Arts/Education  Researcher 

132 

3.5 

University  Student 

113 

3.0 

State/County  Arts  Council  Staff 

84 

2.2 

State/County  Office  of  Ed  Staff 

79 

2.1 

Arts  Evaluator 

73 

1.9 

Cultural  Organization  Staff 

72 

1.9 

Other 

23 

0.6 

Freelance  Artist 

10 

0.3 

Total 

3,744 

100 

The  intention  of  the  study  was  to  collect  information  about  current  trends  and  practices  in  the 
assessment  of  student  learning  in  the  arts  the  at  the  K-12  level  in  the  United  States.  I lowever,  the 
survey  procedures  provided  opportunities  for  people  to  complete  the  survey  even  though  they  were 
not  the  intended  audience  of  the  survey.  For  example,  many  university  students  are  members  of  the 
national  arts  sendee  organization (s)  associated  with  their  major.  Since  the  survey  was  sent  to  all 
members  of  the  partner  organizations,  university7  students  were  sent  access  codes  to  complete  the 
survey.  Statistical  comparisons  and  examinations  of  data  were  made  to  determine  if  groups  of 
respondents  should  be  aggregated,  further  disaggregated,  or  removed  from  the  study. 

The  first  step  was  to  determine  who,  if  any7one,  should  be  removed  from  the  studv.  A review  of  the 
data  revealed  that  university  students  and  university  faculty7  (not  researchers)  responses  were  more 
aligned  with  current  practices  at  the  university  level  rather  than  the  K-12  level.  As  such, 
respondents  who  indicated  they  were  university  students  or  university7  facultv  were  removed  from 
the  study.  Additionally,  data  from  the  ten  freelance  artists  were  reviewed  and  it  was  determined  that 
six  were  responding  as  independent  arts  education  consultants  and  four  were  responding  as 
practicing  artists  (not  teaching  artists).  Thus,  the  six  consultants  were  reclassified  as  arts 
organization  staff  and  the  four  practicing  artists  were  removed  from  the  study. 

The  second  step  was  to  examine  if  any  groups  should  be  disaggregated  or  aggregated  for  purposes 
of  analysis.  Statistical  comparisons  between  arts  researchers  and  arts  evaluators  determined  the  two 
groups  had  similar  response  patterns  and  thus  their  data  could  be  aggregated  into  one  group. 
Similarly,  arts  organization  staff  and  cultural  organization  staff  survey  responses  were  also  very 
similar  so  their  data  were  also  aggregated  into  one  group.  In  contrast,  statistical  comparisons 
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between  school  staff  and  district  staff  determined  very  distinct  group  response  patterns  so  their  data 
were  disaggregated  into  two  separate  groups.  Another  comparison  examined  the  differences 
between  state  and  county  office  of  education  staff  and  state  and  county  arts  council  staff.  The 
review  determined  these  groups’  responses  were  quite  disdnct  and  therefore  should  not  be 
aggregated.  As  such,  findings  from  this  study  were  separated  into  two  primary7  sections: 

• Arts  Assessment  from  the  Perspective  of  District  and  School  - Consisting  of 
staff  from  districts  and  schools 

• Arts  Assessment  from  the  Perspective  of  Policymakers,  Arts  and  Cultural 
Organization  Staff,  and  Researchers  - Consisting  of  state  and  county  office  of 
education  staff,  state  and  county  arts  council  staff,  local  arts  and  cultural  organization 
staff,  and  arts  researchers  and  evaluators 

Exhibit  4 provides  the  revised  response  rates  based  on  primary  role  for  those  included  in  the  study. 
Details  about  the  respondent  characteristics  within  each  of  these  groups  are  further  described  in  the 
findings  section  of  this  report  (Chapter  4). 


Exhibit  4 - Primary  Role  of  Respondents  included  in  the  Study 


Number  of 
Respondents 

Percent 

District  and  School  Perspective 

District  Staff 

90 

2.7 

School  Staff 

2,079 

61.6 

Policymaker,  Arts  and  Cultural  Organization 
Staff,  and  Researcher  Perspective 

State/County  Office  of  Education  Staff 

79 

2.3 

State/County  Arts  Council  Staff 

84 

2.5 

Arts/Cultural  Organization  Staff 

840 

24.9 

Arts  Researchers/Evaluators 

205 

6.1 

Total 

3,377 

100 

QUANTITATIVE  ANALYSIS 

Individual  data  files  were  received  for  each  partner  agency  that  administered  the  survey  to  its 
members/grantees.  Once  received,  the  files  were  examined  for  coding  errors  by  going  through  each 
participant’s  responses  to  determine  if  they  were  valid  and  within  the  intention  of  the  survey7.  One 
area  in  which  data  were  corrected  occurred  when  participants  would  mark  “other”  and  write  in  a 
response  to  the  question.  In  many  cases,  a review  of  their  comments  found  their  answer  fell  into 
one  of  the  pre-existing  response  options.  For  example,  when  asked  about  the  art  forms  addressed 
by  their  school/agency,  some  respondents  marked  “other”  and  wrote  in  “Drama”  even  though 
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“Theater”  was  a pre-existing  response  option.  When  this  occurred,  the  data  were  re-coded  as  the 
pre-existing  response  option  and  the  “other”  response  was  deleted  from  the  file. 

Once  the  data  files  were  “clean”  and  obvious  errors  or  invalid  responses  removed,  the  data  were 
merged  into  a single  data  file.  A secondary  data  validation  process  was  conducted  examining  the 
means  and  ranges  of  the  data  variables  to  remove  any  additional  errors.  This  included  examining 
data  that  resulted  from  “skip  patterns”  on  the  survey  to  ensure  that  only  those  who  should  have 
answered  a question  had  data  in  the  appropriate  cells.  Once  satisfied  that  the  data  were  as  accurate 
as  possible,  independent  samples  t-tests  were  conducted  to  explore  the  possibilities  of  combining 
respondent  groups  for  the  analysis  process  as  described  above.  Frequencies  and  percentages  were 
calculated  for  the  six  final  respondent  groups  as  appropriate  using  SPSS.  Additional  exploratory 
analyses  were  conducted  using  crosstabs  that  allowed  for  visual  comparisons  of  groups  of 
respondents,  such  as  the  types  of  art  forms  addressed  by  the  type  of  respondent  (e.g.,  school  staff, 
office  of  education  staff,  arts  organization  staff,  arts  researcher). 


QUALITATIVE  ANALYSIS 

All  qualitative  responses  were  carefully  coded  and  analyzed.  The  qualitative  analysis  process  closely 
followed  the  analysis  plan  recommended  by  Miles  and  Huberman  (1994)  including  a system  of  data 
reduction,  data  display  or  process/outcomes  matrix,  conclusion  drawing,  and  verification.  The 
process  of  data  reduction  is  intended  to  succinctly  capture  the  data  as  they  relate  to  the  research 
questions.  In  this  case,  the  research  questions  addressed  the  needs  of  the  field  in  assessing  student 
learning  in  the  arts. 

Of  the  3,744  survey  respondents,  3,014  (80.6%)  responded  to  the  qualitative  question  relating  to  the 
needs  of  the  field  for  assessing  student  knowledge,  skills,  and  learning  in  the  arts.  I lowever,  not  all 
responses  could  be  coded.  Responses  that  were  not  clear  or  did  not  address  the  needs  of  the  field 
were  not  coded.  For  example,  some  respondents  used  the  open-ended  questions  as  an  opportunity 
to  share  highlights  about  their  own  program  or  talk  about  the  need  to  improve  access  to  the  arts  for 
all  students.  Other  non-coded  responses  indicated  the  participant  did  not  understand  the  question 
or  was  unsure  how  to  answer  the  question.  Qualitative  data  from  each  group  of  respondents  (office 
of  education  staff,  arts  council  staff,  district  staff,  school  staff,  arts/cultural  organization  staff,  and 
researchers)  were  coded  separately  and  analyzed  for  themes  both  within  and  across  groups. 

Findings  from  the  qualitative  analysis  are  presented  in  Chapter  4. 
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Chapter  3: 
Review  of  the  Literature 
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Literature  Review 


WestEd  staff  conducted  an  extensive  literature  search  tor  materials  and  tools  related  to  the 
assessment  of  student  learning  in  the  arts.  Overall,  more  than  1,000  documents,  books,  articles, 
assessment  tools,  resource  guides,  websites,  and  other  related  materials  were  identified  and  reviewed. 
They  were  categorized  based  on  the  type  of  document:  assessment  tool,  collection  of  assessment 
tools,  resources  describing  a how-to  process,  technical  reports,  informational  materials,  and  “other.” 
Each  piece  of  literature  identified  was  given  initial  ratings  to  determine:  (1)  if  it  was  relevant  enough 
to  the  focus  of  the  study  to  be  included  in  the  review,  and  (2)  if  the  quality  of  the  document  was 
high  enough  to  merit  inclusion  in  the  study.  For  a detailed  description  of  how  relevance  and  quality 
were  determined,  see  the  Methodology  secdon  of  this  report  (Chapter  2).  This  chapter  of  the  report 
first  presents  a brief  overview  of  the  literature  found,  followed  by  detailed  secdons  for  each  tvpe  of 
literature,  including  brief  vignettes  portraying  examples  of  high-  and  low-quality  materials  within 
each  type  of  document. 

Of  all  the  literature  reviewed,  727  separate  items  were  analyzed  from  the  Master  Tracking  database. 
More  than  half  were  classified  as  informational  materials  (371,  51.0%).  The  next  most  common 
types  of  literature  identified  were  individual  assessment  tools  (155,  21.3%),  technical  evaluation  and 
research  reports  (68,  9.4%),  collections  of  assessments  (58,  8.0%),  and  resource  guides  (39,  5.4%). 
Lesson  plans  without  assessments  (25,  3.4%)  and  “Other  materials”  (1 1,  1.5%)  were  also  located 
during  the  search  but  not  included  in  the  study.  As  noted  above,  to  be  included  in  the  content 
analysis  of  the  study,  literature  needed  to  meet  the  relevance  requirements  and  then  receive  a quality 
rating  of  7 or  higher  (see  Chapter  2 — Methodology  for  more  information).  More  than  half  (53.9%) 
of  the  documents  identified  during  the  search  did  not  meet  the  standards  for  relevance.  Examples 
of  why  documents  were  excluded  from  the  study  included: 

• Research  studies  and  evaluation  reports  often  focused  on  learning  through  the  arts 
rather  than  in  the  arts,  such  as  how  arts  integration  programs  improve  thelanguage 
skills  of  English  language  learners 

• Arts  assessments  actually  measured  other  components  of  arts  education  than 
knowledge  or  skills,  such  as  appreciation,  participation,  and  satisfaction 

• The  information  provided  was  designed  and  intended  for  other  countries,  such  as 
frameworks  for  meeting  Australian  or  Canadian  standards 

• The  resource  focused  on  university'  students  rather  than  K-12  students 

• The  site  contained  a variety  of  lesson  plans,  but  no  assessment  tools  were  available  to 
measure  the  impact  of  the  lessons  on  student  knowledge  or  skills  in  the  arts 


This  number  is  lower  than  the  total  number  reviewed  because  sets  of  items  were  captured  as  collections  and  are 
counted  only  once  although  they  represent  multiple  individual  items  reviewed. 
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• The  website  simply  provided  links  to  other  websites  (that  were  also  reviewed  for 
inclusion  in  the  study)  rather  than  any  original  information 

Literature  was  also  excluded  from  the  study  if  it  did  not  meet  the  minimum  requirements  for  quality7. 
As  described  in  the  Methodology  section  (Chapter  2),  quality7  was  gauged  separately  for  different 
types  of  documents  (e.g.,  assessment  tool,  resource,  technical  report)  because  the  user  would  have 
different  expectations  for  each  type.  For  example,  a low-quality  assessment  tool  might  be  a rubric 
that  was  so  subjective  different  raters  would  assign  different  scores,  while  a low-quality7  resource  may 
be  so  complicated  and  missing  key  steps  in  the  process  that  the  user  would  not  be  able  to  follow 
along.  More  detailed  examples  of  high-  and  low-quality  literature  are  provided  as  vignettes  in  the 
sections  below. 

The  proportion  of  literature  that  met  the  requirements  for  both  relevance  and  quality  varied  by 
document  type.  While  informational  documents  were  the  most  often  located  type  of  document, 
65.0%  were  considered  not  relevant  to  the  study  and  41.8%  of  relevant  items  did  not  meet  the 
standards  for  quality7.  Collections  of  assessment  tools  had  the  greatest  proportion  of  documents  that 
met  the  relevance  (87.0%)  standards,  and  the  highest  proportion  of  relevant  items  that  met  the 
quality7  standards  (66.0%).  In  comparison,  87.5%  of  technical  reports  identified  were  not  considered 
relevant  to  the  studyy  and  among  the  eight  relevant  reports  only  three  (37.5%)  were  rated  as  high- 
quality.  Exhibit  5 presents  the  number  of  documents  identified  for  each  type  of  literature,  the 
number  unable  to  be  rated,  how  many  were  removed  due  to  relevance,  how  many  relevant  items 
were  removed  due  to  low  quality,  and  how  many  were  analyzed  as  part  of  the  study. 

Exhibit  5 - Number  of  Items  Reviewed  by  Type  of  Document 


Total 

Items 

Items  Removed 

Relevant  Items 

Relevant,  High- 

Items 

not 

due  to 

Removed  due  to 

Quality  Items 

Identified 

Rated 

Relevance 

Quality 

Reviewed 

n 

n 

n % 

n % 

n % 

Informational  Documents 

371 

22 

227 

65.0 

51 

41.8 

71 

58.2 

Assessment  Tools 

155 

6 

59 

39.6 

60 

66.6 

30 

33.3 

Technical  Reports 

68 

4 

56 

87.5 

5 

62.5 

3 

37.5 

Collections 

58 

4 

7 

13.0 

16 

34.0 

31 

66.0 

Resources 

39 

2 

7 

18.9 

17 

56.7 

13 

43.3 

Lesson  Plans 

25 

0 

25 

100 

N/A 

N/A 

Other 

11 

0 

11 

100 

N/A 

N/A 

Total 

727 

38 

392 

56.9 

149 

50.2 

148 

49.8 

Subject-specific  databases  were  created  for  each  document  type  to  categorize  and  catalogue  the 
relevant  information.  Once  completed,  a content  analysis  of  the  databases  was  conducted.  The 
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following  sections  present  findings  and  summaries  of  the  information  reviewed  during  the  content 
analysis  process. 


ASSESSMENT  TOOLS 


Of  the  155  documents  initially  reviewed  that  were  classified  as  assessment  tools,  6 were  unable  to  be 
rated  and  59  (39.6%)  were  removed  due  to  relevance.  Of  the  remaining  90  relevant  documents,  30 
(33.3%)  met  the  minimum  standards  for  quality.  The  qualifying  assessment  tools  were  written  and 
produced  by  various  agencies  and  individuals,  although  they  were  primarily  from  large-scale  testing 
agencies,  state  education  agencies,  and  university  professors.  The  availability  of  and  access  to  the 
assessments  varied  widely  — 20  needed  to  be  ordered  directly  from  the  publisher  or  author,  two  were 
out  of  publication,  and  two  were  designed  to  be  completed  online.  Cost  information  was  available 
for  18  assessments,  of  which  10  were  available  free  of  charge  and  the  remaining  8 ranged  in  price 
from  $10  to  $90.  Eight  of  the  assessments  were  publicly  available,  either  as  free  downloads  from  the 
Internet  (n  = 4)  or  for  purchase  (n  = 4). 

Four  assessments  covered  multiple  art  forms  while  the  other  26  were  tailored  to  a single  art  form  or 
even  more  specific  art  forms  such  as  watercolor  or  modern  dance.  As  might  be  expected,  visual  arts 
and  music  were  the  most-represented  art  forms,  with  15  and  1 1 tools  respectively.  In  contrast, 
dance,  literary  arts,  and  theater  each  had  three  assessment  tools  included  in  the  study.  Exhibit  6 
presents  an  overview  of  the  number  of  assessments  that  addressed  each  art  form. 


Exhibit  6 - Assessment  Tools  by  Art  Form 


Number  of 
Assessments 

Visual  Arts 

15 

Music 

11 

Dance 

3 

Literary  Arts 

3 

Theater 

3 

Media  Arts 

1 

n = 30  although  4 assessments  assessed  more  than  one  art  form 

Assessments  were  also  categorized  based  on  their  grade-level  audience  (e.g.,  elementary  school, 
middle  school,  high  school),  although  some  assessments  had  different  versions  based  on  grade.  For 
example,  a state  education  agency  may  administer  its  arts  assessment  test  in  grades  5,  8,  and  1 1 using 
different  versions,  but  the  assessment  was  only  counted  once  in  the  study.  Slightly  more 
assessments  were  appropriate  or  had  versions  for  high  school  use  than  those  intended  for 
elementary7  or  middle  school  students  (Exhibit  7),  due  in  part  to  the  Advanced  Placement  and 
International  Baccalaureate  assessments  that  are  only  available  at  the  high  school  level.  Eleven  of 
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the  assessments  indicated  they  were  appropriate  or  had  versions  for  multiple  grade  level  ranges, 
including  five  that  covered  all  three  grade  level  ranges. 


Exhibit  7 — Assessment  Tools  by  Grade  Level 


Number  of 
Assessments 


Elementary  School  (K-5)  14 

Middle  School  (6-8)  14 

High  School  (9-12)  18 


n = 30  although  1 1 assessments  addressed  multiple  grade  level  ranges 


Documentation  from  seventeen  of  the  assessments  indicated  the  tool  tested  student  knowledge  in 
the  arts,  and  20  tested  student  skills.  Of  the  20  skills  assessments,  14  (70.0%)  involved  rubrics. 
Rubrics  were  primarily  analytic  (12  assessments)  with  only  two  being  holistic  in  nature.  ^ 


Eleven  of  the  30  assessments  (36.7%)  were  standards-based,  addressing  primarily  state  standards 
with  only  three  addressing  the  national  standards.  Administration  procedures  ranged  from  large 
group  paper-and-pencil  tests  to  individual,  one-on-one  performance  and  portfolio  reviews.  Twelve 
(40.0%)  of  the  assessments  were  intended  for  classroom  use,  while  others  were  used  as  statewide 
achievement  tests,  exit  exams,  and  measures  for  adjudication. 

In  general,  the  assessment  tools  that  met  the  standards  for  quality  were  developed  by  testing 
agencies,  often  in  collaboration  with  state  education  agencies.  They  were  designed  to  measure 
student  achievement  on  a larger  scale  and  significant  time  and  effort  were  dedicated  to  their 
development.  The  following  is  an  example  of  a high-quality  assessment  tool. 


^ A holistic  rubric  provides  one  rating  or  score  for  the  overall  grade  of  an  item  or  performance.  An  analytic  rubric  . 
provides  multiple  ratings  based  on  multiple  criteria  outlined  in  the  rubric.  For  example,  a visual  arts  rubric  may  include 
criteria  for  color,  technique,  shape,  space,  and  use  of  materials. 
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Example  of  a High-Quality  Assessment  Tool  at  the  State  Level 

The  state  education  agency  contracted  with  a testing  agency  to  develop  this  high-quality 
assessment  tool.  It  has  a clearly  defined  focus  and  audience;  it  is  designed  to  assess  fifth 
grade  students’  knowledge  of  music.  The  tool  is  aligned  to  the  state  standards  and  is 
administered  by  classroom  teachers  in  conjunction  with  other  state  testing  for  Plnglish 
language  arts,  mathematics,  and  science.  The  assessment  tool  contains  multiple-choice 
questions  covering  a wide  variety  of  music  topics,  such  as  identifying  a family  of 
instruments,  selecting  the  type  of  music  associated  with  a particular  culture,  choosing  the 
correct  definition  of  terms,  and  answering  questions  based  on  reading  musical  notes.  The 
development  of  the  assessment  tool  included  checks  to  ensure  high  reliability  and  validity  of 
the  instrument. 


Although  many  locally  developed  assessment  tools  tended  to  score  lower  on  the  quality  ratings, 
some  met  the  criteria  for  inclusion  in  the  study.  These  were  often  developed  by  university 
professors  and  intended  for  high  school  use.  The  following  is  an  example  of  a high-quality 
assessment  tool  designed  at  the  local  level  intended  for  classroom  use. 


Example  of  a High-Quality  Assessment  Tool  at  the  Classroom  Level 

This  assessment  tool  was  designed  by  a university'  professor  to  assess  the  painting  skills  of 
students  in  seventh  through  twelfth  grade.  The  rubric  includes  seven  separate  criteria  that 
measure  artistic  skills  as  well  as  other  components  of  the  assignment,  such  as  composition, 
technique,  use  of  materials,  experimentation,  and  critique.  While  specific  to  painting,  the 
rubric  is  general  enough  to  be  used  to  grade  multiple  projects  because  it  focuses  on  art  skills 
rather  than  the  specifics  of  a single  assignment.  The  ratings  are  on  a five-point  scale  with 
clear  criteria  defining  the  low,  mid,  and  high  ratings.  The  rubric  includes  two  columns  for 
scoring,  one  to  be  used  by  the  student  as  a self-assessment  and  one  to  be  used  by  the 
teacher.  The  design  of  the  tool  indicates  the  potential  for  providing  formative  feedback  to 
students  in  addition  to  a summative  score.  Note  — This  rubric  was  not  identified  during  the 
literature  search,  but  was  provided  to  the  study  team  via  email. 


More  than  one-third  (38.1%)  of  assessment  tools  were  removed  from  the  study  because  they  did  not 
meet  the  criteria  for  relevance.  Often,  what  was  touted  as  an  arts  knowledge  or  skill  assessment 
actually  measured  something  other  than  knowledge  or  skills,  such  as  arts  appreciation,  program 
satisfaction,  engagement,  or  classroom  management.  Two-thirds  of  the  relevant  assessment  tools 
were  considered  to  have  inadequate  quality.  A common  theme  among  the  low-quality  tools  was  a 
lack  of  understanding  of  what  is  and  is  not  a rubric.  Many  documents  were  titled  rubrics,  but  were 
merely  checklists  or  things  to  look  for  during  a performance  without  any  clear  criteria  that 
differentiate  one  score  from  another,  which  is  a key  component  of  a rubric.  This  points  to  a need  in 
the  field  for  professional  development  as  to  what  constitutes  a rubric,  how  it  is  used,  how  it  is 
created,  and  what  components  are  necessary'  to  develop  or  select  a high-quality'  rubric.  Below  is  an 
example  of  an  assessment  tool  that  was  removed  from  the  study  due  to  quality. 
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Example  of  a Low-Quality  Assessment  Tool 


This  assessment  tool  was  designed  by  a university  professor  to  assess  the  visual  art  skills  of 
fifth  grade  students;  however,  the  tool  contains  some  serious  flaws.  Although  it  is  titled  as  a 
rubric,  it  does  not  meet  the  criteria  for  a rubric.  The  tool  simply  provides  a few  questions 
within  each  set  of  criteria  and  raters  answer  the  questions  by  checking  good,  average,  or 
needs  improvement.  It  is  more  aligned  with  a simple  checklist  than  a rubric.  Further,  the 
tool  does  not  actually  assess  art  skills,  but  rather  areas  such  as  assignment  completion,  work 
habits,  creativity,  and  growth  from  previous  assignments.  The  tool  is  extremely  subjective 
since  there  are  no  clear  definitions  of  the  criteria,  and  the  questions  are  so  generic  that  it 
could  be  used  with  any  subject.  Although  touted  as  an  arts  rubric,  this  tool  is  not  a rubric 
nor  does  it  adequately  assess  any  art  form. 


In  sum,  a variety  of  assessments  met  the  minimum  standards  of  quality  and  relevance  needed  for  the 
content  analysis  process.  The  30  assessment  tools  were  developed  by  various  authors/publishers, 
measured  a range  of  art  forms  and  grade  levels,  and  were  administered  in  a variety  of  ways  (e.g., 
individual  test,  group  test,  individual  performance,  group  performance).  However,  many  more  low- 
quality  and  less-useful  assessments  to  measure  student  learning  in  the  arts  are  easily  accessible  and 
available  through  the  Internet  and  other  sources,  exposing  a need  to  promote  the  development  and 
dissemination  of  high-quality  assessment  tools.  Lastly,  all  assessment  tools  should  be  reviewed 
carefully  to  ensure  a proper  fit  for  the  knowledge  and/or  skills  to  be  assessed. 


COLLECTIONS  OF  ASSESSMENT  TOOLS 


Collections  were  defined  as  a group  of  four  or  more  assessment  tools.  Here,  the  percentage  of  high- 
quality,  relevant  documents  was  higher  than  when  reviewing  single  assessment  tools.  Of  the  58 
collections  identified,  four  were  not  rated  because  of  access  issues  and  7 (13.0%)  were  removed  due 
to  lack  of  relevance  to  the  study.  Of  the  remaining  47  collections,  16  (34.0%)  were  considered  low- 
quality,  leaving  31  (66.0%)  relevant,  high-quality  collections.  Twenty  of  these  31  collections  were 
sets  of  assessments,  such  as  websites  maintained  by  a state  education  agencies,  national  art 
organizations,  or  school  districts.  Seven  of  the  collections  were  sets  of  assessment  tools  with  lesson 
plans,  such  as  website  designed  for  teachers  to  upload  and  share  lessons.  Three  were  sets  of 
assessments  provided  with  curricula  developed  by  textbook  publishers,  and  one  collection  of 
measures  was  disseminated  through  a journal  article. 

The  high-quality  collections  varied  widely  from  concise  sets  of  a few  targeted  tools  created  by 
publishing  companies  or  state  education  agencies,  to  vast  databases  containing  hundreds  of 
individual  assessments  created  and  uploaded  exclusively  by  classroom  teachers.  For  collections 
containing  30  or  fewer  assessments,  the  evaluation  team  reviewed  each  individual  assessment  tool 
and  provided  a single  quality  rating  across  the  collection.  For  collections  containing  more  than  30 
assessments,  the  evaluation  team  reviewed  a random  sample  of  assessments  and  gleaned  pertinent 
information  from  the  sample. 
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Contributors  to  high-quality  collections  included  authors/editors  associated  with  publishing 
companies,  classroom  teachers  and  teaching  artists,  agency  and  organization  members,  and  state 
education  agencies.  The  larger,  less-structured  collections  tended  to  be  comprised  of  contributions 
from  classroom  teachers.  Twenty-six  of  the  collecuons  (83.9%)  were  available  free  to  the  public, 
with  24  of  them  available  via  Internet  download.  Other  formats  for  collections  of  assessments 
included  books  (n  = 3),  online  availability  without  download  (n  = 3),  and  in-person  use  only  (n  = 1). 

Music  was  the  most  common  art  form  addressed  by  the  high-quality7  collections  of  assessments,  with 
13  of  the  collections  including  music  assessments.  This  was  followed  by  visual  arts  with  1 1 
collections  and  theater  with  8 assessment  collections.  Dance  assessments  were  represented  by  seven 
of  the  collections,  while  two  contained  literary  arts  assessments  and  two  contained  media  arts 
assessments.  Eleven  of  the  collections  also  provided  assessments  for  more  specific  art  forms,  such 
as  concert  band,  piano,  drawing,  and  choral  performances. 

High-quality  assessment  collections  were  also  categorized  based  on  the  grade  levels  addressed  by  the 
assessment  tools.  As  can  be  seen  in  Exhibit  8,  most  collections  included  assessment  tools  for 
multiple  grade  levels,  and  slightly  more  covered  high  school  grades  than  middle  school  and 
elementary7  school  grades;  however,  it  is  unknown  if  this  indicates  more  assessment  occurs  at  the 
high  school  level. 


Exhibit  8 — Collections  of  Assessments  by  School  Level 


Number  of  Collections 


Elementary  School  (K-5) 

21 

Middle  School  (6-8) 

22 

High  School  (9-12) 

28 

n — 31  although  20  collections  contained  assessments  for  use  across  multiple  grade  ranges 


Twenty-four  of  the  high-quality7  collections  contained  assessment  tools  measuring  student 
knowledge  in  the  arts,  and  all  measured  student  skills  in  the  arts  because  they  all  contained  at  least 
one  rubric  designed  to  measure  art  skills.  Other  assessment  types  included  within  the  collections 
were  portfolio  assessments,  checklists,  paper/pencil  tests,  and  written  response.  Twentv-one  of  the 
collections  (67.7%)  contained  standards-based  assessments,  addressing  state  (n  = 14)  and  national  (n 
= 9)  standards. 

Of  the  58  items  initially  identified  that  were  classified  as  collections  of  assessments,  more  than  half 
(53.4%)  met  the  minimum  standards  for  relevance  and  quality;  the  highest  proportion  of  any 
document  type.  Two  key  factors  led  to  this  higher  proportion:  (1)  professional  textbook  publishers 
that  included  assessments  with  their  materials,  which  tended  to  be  of  higher  quality,  were 
represented  in  this  category7;  and  (2)  some  organizations  that  collected  assessment  tools  vetted  the 
tools  for  quality  prior  to  inclusion  in  the  collection.  The  following  is  an  example  of  a high-quality 
collection  included  as  part  of  an  arts  curriculum. 
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Example  of  a High-Quality  Collection 


This  high-quality  collection  is  part  of  a visual  arts  curriculum  series  developed  by  a textbook 
publisher  that  includes  assessments  for  each  unit  of  study.  The  assessments  are  criterion- 
referenced  and  aligned  to  national  and  state  standards.  The  publisher  offers  separate 
curricula  and  textbooks  for  each  grade  level  from  kindergarten  through  high  school,  with 
upper  grades  including  more  extensive  concentrations  in  more  specific  art  forms  such  as 
drawing,  painting,  design,  sculpture,  photography,  and  printmaking.  The  teacher  guides 
contain  information  related  to  formative  assessment,  including  questions  that  can  be  asked 
during  lessons.  In  addition,  summative  assessments  for  use  as  end-of-unit  exams  are  also 
available  and  include  multiple-choice,  matching,  fill-in,  and  open  response  questions.  This 
curriculum  series  is  both  comprehensive  and  well-designed,  and  includes  a vast  number  of 
assessments  and  assessment-related  activities. 


High-quality  collections  came  from  a variety  of  sources.  In  addition  to  assessments  included  with 
textbooks,  many  oi  the  high-quality  collections  were  from  organizations  that  developed  systems  to 
review  submitted  assessment  tools  to  ensure  they  met  minimum  quality  standards.  One  collection 
included  assessments  within  a larger  journal  article: 


Example  of  a High-Quality  Collection 


This  collection  of  dance  assessments  is  embedded  within  a journal  article  on  authentic 
assessment  written  by  a state  education  agency  staff  member.  Of  the  seven  skills  rubrics 
presented,  four  are  holistic  (assessing  lyricism,  bravura  quality,  technical  proficiency,  and 
dance  critique)  and  three  are  analytic  (measuring  movement  skills,  physical  skills,  and 
creativity).  Criteria  addressed  in  the  analytic  rubrics  include  technical  proficiency,  rhythmic 
acuity,  musicality,  coordination,  and  use  of  space.  All  rubrics  in  this  collection  are  aligned  to 
national  dance  standards  and  are  appropriate  for  use  with  high  school  students.  In  addition, 
the  creativity  rubric  contains  only  one  dance-specific  measure  and  would  be  very  easy  to 
modify  for  other  art  forms. 


As  noted  earlier,  approximately  one-third  (34.0%)  of  collections  relevant  to  the  study  were  found  to 
be  of  low  quality.  Most  of  these  collections  allowed  teachers  and  other  educators  to  upload 
assessment  tools  (some  with  corresponding  lesson  plans)  that  were  made  public  with  no  review 
process.  While  this  occurred  with  many  open  websites,  it  also  was  found  with  some  larger  agencies 
and  state  education  agencies. 
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Example  of  a Low-Quality  Collection 


This  collection  of  assessment  tools  was  compiled  and  presented  as  part  of  an  online  teacher 
handbook  on  arts  assessment.  Sample  assessment  tools  are  provided  for  dance,  drama, 
literary  arts,  music,  and  visual  arts.  Although  a few  of  the  included  assessments  are 
acceptable  — case  in  point,  an  analytic  writing  rubric  for  literary  arts  — the  majority  do  not 
meet  the  quality  benchmark.  For  example,  one  theater  rubric  scores  students  on  a five- 
point  scale  with  no  descriptors  of  the  criteria  or  gradations  of  quality.  Another  assessment 
is  labeled  as  a music  rubric,  but  is  a simple  yes/ no  checklist  of  student  mastery  of  various 
performance  tasks.  Of  concern  is  the  fact  that  the  arts  framework  development  committee 
presents  these  samples  as  a guide  and  resource  for  teachers  and  encourages  them  to  use 
these  low-quality  assessment  tools  as  a starting  place  for  designing  their  own  assessments. 

In  contrast  to  high-quality  collections  identified  trom  other  states,  in  this  case  the  arts 
framework  development  committee  did  not  have  an  adequate  system  in  place  to  properly 
vet  materials  to  ensure  onlv  high-quality'  assessments  were  made  available  to  the  teachers. 


The  quality  of  collections  of  assessments  appear  to  generally  involve  two  factors:  (1)  agencies  willing 
to  put  in  the  time  and  resources  needed  to  develop  high-quality  tools,  and  (2)  agencies  willing  to  collect 
assessment  tools  and  implement  procedures  to  review  them  and  ensure  that  onlv  the  higher-quality 
tools  are  made  available.  As  such,  arts  teachers,  teaching  artists,  researchers,  and  others  looking  to 
these  large  collections  of  assessment  tools  should  take  steps  to  identify  how  assessment  tools  are 
uploaded  to  the  collection  and  what,  if  any,  review  process  is  undertaken  to  ensure  the  quality'  and 
validity  of  the  assessments. 


RESOURCES 


Resources  were  defined  as  “how-to”  guides  that  provided  readers  with  instructions  on  how  to 
complete  a task,  such  as  developing  assessment  tools  or  implementing  and  administering 
assessments.  An  example  of  a resource  would  be  a step-by-step  guide  on  how  to  develop  a rubric  to 
assess  student  skills  in  the  arts. 

Of  the  39  documents  initially  reviewed  that  were  classified  as  resource  guides,  two  were  unable  to  be 
rated  and  seven  (18.9%)  were  removed  due  to  lack  of  relevance  to  the  study.  Of  the  remaining  30, 

17  (56.7%)  were  characterized  as  low-quality  and  13  (43.3%)  met  the  minimum  quality  standard. 
These  13  higher-quality  resources  were  written  or  published  by  a variety  of  organizations,  including 
state  education  agencies,  a state  art  council,  a national  arts  organization,  a large  school  district,  and  a 
textbook  publisher,  all  larger  entities  capable  ot  dedicating  the  time  and  effort  needed  to  develop 
such  a product.  Five  of  the  thirteen  were  books,  two  were  item  banks  with  questions  available  for 
use,  two  were  journal  articles,  two  were  how-to  websites,  and  one  was  a professional  development 
workshop.  Ten  of  the  resources  had  cost  information  available  — eight  were  free  to  access  and 
download  from  the  Internet,  one  cost  $100  from  the  publisher,  and  the  professional  development 
workshop  ranged  from  $250-$450.  Ten  of  the  resources  were  intended  for  teacher  use,  compared  to 
two  intended  for  local  arts  organizations  (Exhibit  9). 
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Exhibit  9 — Intended  Audience(s)  for  Resources/How-to  Guides 


Number  of  Resources 

Teachers 

10 

School  and/or  School  District  Staff 

6 

State  Education  Agency  Staff 

3 

Arts  Organization  Staff  2 

Students 

n = 13  although  6 resources  had  2 or  more  intended  audiences 

Some  of  the  high-quality  resources  were  targeted  for  one  specific  art  form,  while  others  applied  to 
multiple  art  forms.  For  example,  6 of  the  13  high-quality  resources  provided  general  information 
that  could  be  used  with  any  art  form,  such  as  a guide  for  developing  rubric  assessments  for  use  in 
the  classroom.  Three  of  the  resources  provided  information  specific  to  music  and  visual  arts, 
another  two  were  tailored  specifically  to  music,  one  was  tailored  specifically  to  visual  arts,  and  one 
was  tailored  specifically  to  dance. 

The  high-quality  resources  also  covered  a variety'  of  assessment  and  data  use  topics,  such  as  creating 
assessments,  collecting  data,  or  a general  how-to  guide  for  student  assessment  in  the  arts.  Exhibit  10 
presents  the  number  of  resources  that  covered  each  topic  area. 

Exhibit  10  - Topics  Discussed  in  Resource  Documents 


Number  of  Resources 


General  Information  and  How-to  on  Assessment  10 

Creating  a Skills  Assessment  9 

Creating  a Knowledge  Assessment  7 

General  How-to  on  Using  Data  5 

Selecting  a Skills  Assessment  3 

Selecting  a Knowledge  Assessment  2 

Collecting  Data  2 

Analyzing  Data  2 

Locating  an  Existing  Knowledge  Assessment  1 

Locating  and  Existing  Skills  Assessment  1 


n = 13  although  9 resources  covered  more  than  one  topic 


A substantial  level  of  effort  went  into  the  resources  that  met  the  relevance  and  quality  requirements. 
The  higher  quality  resources  tended  to  be  from  large  agencies  with  the  knowledge  and  capacity  to 
produce  such  guides.  Two  examples  of  high-quality  resources  were: 


1 
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Example  of  a High-Quality  Resource  Tool 


This  series  of  charts  was  designed  by  a textbook  publisher  to  help  teachers  improve  the 
assessment  of  student  work  in  visual  arts.  Although  aimed  at  drawing,  the  resource  helps 
teachers  better  understand  the  characteristics  of  rubrics  and  can  be  used  as  the  basis  for 
other  visual  art  forms.  The  five  charts  cover  visual  field,  shape,  value,  dimension,  and  light. 
Each  topic  is  further  broken  down  into  a number  of  criteria  — for  example,  light  is  judged 
on  light  & shadow,  texture,  reflection,  and  transparency.  For  each  criterion,  the  authors 
provide  a definition  of  terms  and  a list  of  artistic  components  that  affect  the  ratings. 

Ratings  are  on  a 20-point  scale  grouped  into  four  levels,  each  accompanied  by  an  actual 
student  drawing  with  an  explanation  of  the  rating  within  the  framework  of  the  artistic 
components.  The  drawings  for  each  level  provide  users  with  a visual  depiction  of  the 
rating  and  the  typical  characteristics  associated  with  products  rated  at  that  level.  A teacher’s 
guide  is  available  to  enhance  teachers’  use  of  the  charts,  and  the  charts  can  be  posted  on 
walls  to  provide  students  with  clear  representations  of  how  a drawing  rated  at  level  one 
differs  from  a drawing  rated  a level  two,  three,  or  four. 


Example  of  a High-Quality  Resource  Publication 


This  comprehensive  guide,  developed  by  a statewide  coalition  of  county  office  of  education 
staff,  provides  a review  of  literature  on  arts  assessment  and  includes  references  for 
educators  to  use  when  designing  and  implementing  both  classroom-based  and  district-level 
arts  assessments.  It  begins  by  answering  general  questions  such  as  why  arts  assessment  is 
important  and  the  qualities  of  good  arts  assessment,  then  debunks  several  myths  of  arts 
assessment  including  the  notion  that  arts  is  entirely  subjective  or  that  art  is  all  about  the  end 
product.  The  heart  of  this  toolkit  is  the  step-by-step  procedures  and  tasks  provided  to 
assist  districts  in  creating  a districtwide  arts  education  assessment  system,  including 
establishing  a planning  committee,  setting  goals  and  objectives,  connecting  assessment  to 
standards  and  instruction,  establishing  parameters  for  the  assessment  system,  and  creating 
an  implementation  timeline.  Assessment  tools  such  as  rubrics,  observation  protocols,  and 
portfolios  are  discussed,  with  links  to  additional  information  and  resources.  Lastly,  the 
document  highlights  dance,  music,  theater,  and  visual  arts  and  examines  discipline-specific, 
standards-based  arts  assessment  efforts  both  within  the  state  and  nationwide. 


By  contrast,  many  of  the  lower-quality  resources  tended  to  be  web-based  compilations  of 
information  rather  than  printed  guides,  which  may  be  an  indication  that  those  willing  to  produce 
hard  copy  materials  are  more  likely  to  also  dedicate  the  time  and  energy  needed  to  develop  high- 
quality  content.  The  following  is  an  example  of  a low-quality  resource. 
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Example  of  a Low-Quality  Website  Resource 


This  web-based  resource  presents  information  about  electronic  portfolios.  The  author 
defines  terms  and  discusses  the  use  of  electronic  portfolios,  i.e.,  information  stored  on  a 
computer  hard  drive,  compact  disc,  or  other  electronic  media  rather  than  hard  copy. 
However,  the  information  and  steps  provided  are  lacking  sufficient  detail  to  be  a trulv  useful 
resource  for  arts  educators.  Although  some  of  the  material  could  be  of  interest  to  arts 
educators,  particularly  those  in  visual  arts,  it  is  not  relevant  to  the  assessment  of  student 
knowledge  and  skills  in  the  arts  and  thus  is  not  considered  to  be  a high-quality  arts 
assessment  resource. 


How-to  resources  can  be  valuable  assets  to  individuals  and  organizations  looking  to  implement  new 
practices,  such  as  developing  assessment  tools  or  designing  a program  evaluation.  However,  56.7% 
of  relevant  resources  did  not  meet  the  standards  for  quality',  and  among  those  that  met  the  standards 
only  two  were  designed  for  arts  organization  staff.  The  field  is  looking  to  implement  and  improve 
assessment  practices,  and  needs  high-quality,  useful  resources  to  aid  in  that  endeavor. 


INFORMATIONAL  DOCUMENTS 


Informational  documents  were  defined  as  those  that  provided  educational  or  instructional 
information  about  assessing  knowledge  and  skills  in  the  arts.  This  included  overviews,  definitions  of 
terms,  policies,  standards,  and  detailed  descriptions  of  student  assessment  in  the  arts.  Examples  of 
informational  documents  included  state  frameworks  for  arts  education,  books  that  presented 
different  tvpes  of  assessments  and  how  they  are  used,  and  articles  that  discussed  theoretical 
approaches  to  assessment  of  arts  education. 

Such  documents  were  voluminous  in  the  database.  Of  the  371  informational  documents  identified 
through  the  literature  search,  22  were  unable  to  be  reviewed  and  227  (65.0%)  were  removed  because 
they  were  not  relevant  to  the  current  study.  Of  the  remaining  122  documents,  71  (58.2%)  met  the 
standards  for  high  quality.  Of  these,  40  were  developed  by  a single  source  — a state  that  developed  a 
separate  guide  for  nearly  every  grade  level  across  the  four  major  art  forms.  For  the  purposes  of  this 
analysis,  these  items  were  collapsed  into  four  separate  items,  one  for  each  art  form,  resulting  in  35 
high-quality  informational  documents  reviewed  in  detail. 

These  high-quality  documents  were  authored/published  by  a wide  variety  of  organizations,  including 
national  arts  organizations,  state  education  agencies,  state  arts  councils,  institutions  of  higher 
education,  school  districts,  textbook  publishers,  educational  research  agencies,  and  local  arts 
organizations.  Ten  of  the  documents  reviewed  were  informational  websites,  seven  were  books,  two 
were  journal  articles,  and  16  fell  into  the  “other”  category,  which  included  PDF  tiles  available  for 
download  and  documents  produced  for  a specified  audience  such  as  state  policies,  frameworks,  and 
standards  intended  for  classroom  teachers  within  that  state. 
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Some  documents  were  general  enough  to  he  relevant  to  all  audiences,  while  others  were  targeted  to  a 
single  audience.  For  example,  all  35  documents  included  information  relevant  to  classroom 
teachers,  and  six  of  the  documents  were  specifically  intended  for  classroom  teachers.  Twenty-eight 
of  the  documents  were  also  relevant  to  school-  and/or  district-level  administrators  and  18 
documents  included  information  relevant  to  grantees  such  as  local  arts  organizations.  Exhibit  1 1 
presents  the  number  of  documents  intended  for  the  various  audiences. 

Exhibit  11  - Intended  Audience(s)  for  High-Quality  Informational  Documents 


Number  of  Documents 


Teachers 

35 

School  and/or  School  District  Administrators 

28 

Grantees/  Arts  Organization  Staff 

18 

General  Population 

10 

State  Education  Agency  Staff 

9 

Research  Agency  or  University  Staff 

6 

n — 35  although  29  had  2 or  more  intended  audiences 


The  majority  of  the  informational  documents  (23  out  of  35,  65.7%)  were  either  generic  enough  to 
cover  all  art  forms  or  included  separate  sections  that  covered  dance,  music,  theater,  and  visual  arts. 
Of  the  remaining,  three  covered  dance,  three  covered  music,  three  covered  visual  arts,  and  two 
covered  theater.  None  of  the  documents  specifically  targeted  literary  arts,  media  arts,  or  more 
specific  art  forms  (e.g.,  ceramics,  jazz,  poetry). 

The  informational  documents  covered  a large  variety  of  topics,  and  nearly  all  covered  muldple 
topics.  Twenty-seven  of  the  35  included  information  on  classroom-based  assessment  practices,  such 
as  how  and  when  to  administer  assessments,  or  information  on  developing  classroom  assessments. 
Twenty-four  of  the  documents  contained  information  about  performance-based  assessments,  such 
as  what  performance-based  assessment  means,  how  performance-based  assessments  should  be  used, 
and  how  staff  can  grade  or  score  performance-based  assessments.  Exhibit  12  presents  the  number 
of  high-quality  informational  documents  that  included  at  least  a section  on  each  of  the  following 
topics. 
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Exhibit  12  - Topics  Discussed  in  Informational  Documents 


Number  of  Document 


Classroom  Assessment  27 

Performance-based  Assessment  24 

General  Assessment  Information  20 

Evaluating  Lessons/Classroom  Practices  18 

Portfolio  Assessment  12 

Program  Evaluation  11 

Formative  Assessment*  11 

Authentic  Assessment  9 

Evaluating  Assessment  Tools  and  Measures  8 

Other  7 

Defining  Goals  and  Objectives  6 

State-level  Facts  or  Information  6 

Formative  Evaluation*  5 

Standardized  Assessment  4 

Policy  Documents  4 

Collaboration  1 


n — 35  although  31  documents  contained  information  on  two  or  more  topic  areas 

* Note:  Formative  assessment  is  a process  used  during  instruction  that  provides  feedback  to  adjust  ongoing  teaching  and 
learning  (Heritage,  2010),  while  formative  evaluation  is  typically  conducted  for  improvement  of  a program  or  product 
(Scriven,  1991). 

The  complexity  of  informational  documents  varies  based  on  the  intended  audience.  For  example,  a 
document  aimed  at  improving  teachers’  use  of  research  may  describe  what  reliability  and  validity  of 
an  assessment  means  and  why  it  is  important,  while  a document  aimed  at  researchers  may  describe 
the  formulas  and  statistical  considerations  when  calculating  reliability  and  validity7.  Informational 
documents  can  serve  as  a form  of  professional  development  when  a clear  audience  is  defined  and 
the  level  of  detail  is  aligned  with  their  current  knowledge  and  needs.  The  following  is  an  example  of 
a high-quality  information  website. 
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Example  of  a High-Quality  Informational  Website 


This  website  includes  a section  on  performance  standards  and  assessment  strategies  in 
music.  Assessment-related  topics  include  program  evaluation,  classroom  evaluation, 
performance-based  assessment,  and  general  assessment  information  (such  as  a glossarv  of 
related  terms).  The  assessment  strategies  section  includes  information  on  procedures, 
challenges  and  limitations,  and  ways  of  reporting  results.  In  addition,  the  website  contains 
sections  grouped  by  grade  level  that  provide  the  national  content  standards  for  music  and 
offer  achievement  standards,  assessment  tasks,  and  possible  response  descriptions  for  each. 
The  content  is  written  at  a basic  level  and  progresses  to  a more  moderate  level,  making  it 
appropriate  for  a wide  range  of  audiences,  including  teachers,  administrators,  and  local  arts 
organization  staff. 


While  several  high-quality  information  documents  emerged  from  the  review,  one  particular  book 
stood  out  because  of  its  comprehensive  information  related  to  assessment. 


Example  of  a High-Quality  Informational  Publication 


This  book  was  published  by  a national  arts  education  organization  and  focuses  on  a wide 
range  of  topics  related  to  arts  assessment.  It  describes  in  detail  the  process  of  arts 
assessment,  beginning  with  asking  why  we  should  assess  learning  in  the  arts,  who  should  do 
the  assessing,  and  who  should  be  assessed.  The  next  section  covers  designing  measureable 
goals  and  identifying  experiences  and  assessment  methods  to  measure  them,  concluding 
with  samples  of  assessment  practices  at  various  grade  levels.  A detailed  discussion  of 
assessment  tools  follows,  including  procedures  for  development  and  modification  of  both 
traditional  and  alternative  measures  as  well  as  information  on  administration  and  scoring. 
The  book  concludes  with  a section  on  summarizing  and  reporting  results,  including  the 
importance  of  ethics  in  assessment.  It  is  appropriate  for  a wide  range  of  audiences, 
including  teachers,  administrators,  and  grantees. 


As  noted  earlier,  a majority  (65.0%)  of  informational  documents  initially  identified  were  removed 
from  consideration  because  they  were  not  relevant  to  the  current  study.  This  included  materials  that 
described  the  implementation  of  an  arts  program  that  may  indicate  they  assess  student  learning,  but 
provided  no  information  about  the  assessment  process;  documents  that  pertained  to  other  countries; 
websites  that  discussed  the  importance  of  standards  with  no  reference  to  assessment;  or  articles  that 
describe  methods  for  assessing  the  teaching  skills  rather  than  student  learning.  Of  the  remaining 
122  informational  documents,  51  (41.8%)  failed  to  meet  a minimum  quality  standard,  such  as  a 
website  too  brief  and  basic  to  be  of  use  or  a document  that  only  presents  vague  information  about 
student  assessment.  The  following  is  an  example  of  a low-quality  information  website. 
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Example  of  a Low-Quality  Informational  Website 


Developed  and  maintained  by  a university  professor,  this  website  includes  minimal 
information  related  to  the  assessment  of  student  learning  in  music.  One  link  contains 
downloadable  blank  forms  that  can  be  used  to  create  observational  records  for  in-class 
music  performances;  another  offers  templates  for  creating  Likert-type  scales,  but  neither 
provide  criteria  or  suggestions  for  questions.  In  addition,  the  secdon  of  the  site  devoted  to 
sample  tools  leads  to  a single  rubric  which  is  poorly  constructed.  While  some  material  could 
be  of  interest  to  music  instructors,  such  as  a glossary  of  terms  and  a handful  of  links  to  other 
assessment  sites,  the  portion  of  this  site  dedicated  to  assessing  student  knowledge  and  skills 
in  the  arts  is  too  basic  to  be  useful. 


Informational  documents  can  serve  as  a form  of  professional  development  when  they  present 
relevant  content  in  an  accessible  format.  Such  documents  can  introduce  people  to  new  topics  or 
offer  greater  levels  oi  detail  for  those  who  already  have  a basic  understanding.  As  such,  these 
documents  must  be  accurate  and  provide  a sufficient  level  of  detail  to  enhance  understanding  on  the 
topic.  Informational  documents  were  the  most  common  type  of  document  identified  during  the 
literature  search,  representing  51.0%  of  all  the  materials  located.  However,  65.0%  were  considered 
not  relevant  to  the  study  and  41.8%  of  relevant  documents  did  not  meet  the  standards  for  quality. 
Being  such  an  important  component  of  professional  development,  high-quality  informational 
documents  need  to  be  easily  accessible  and  widely  disseminated  to  ensure  the  field  has  a common 
understanding  on  critical  topics. 


TECHNICAL  REPORTS 


Technical  reports  were  defined  as  research  study  and  evaluation  reports  that  included  the  assessment 
of  student  knowledge  and/or  skills  as  an  outcome  variable.  An  example  of  a qualifying  document 
would  be  the  final  evaluation  report  produced  by  an  arts  organization  and/ or  external  evaluator  as  a 
funding  requirement  for  a grant.  It  should  be  noted  that  such  reports  are  often  not  made  public, 
and  thus  not  part  of  the  database  for  this  study.  When  a tunder  requires  an  evaluation  as  a 
condition  of  the  grant,  the  resulting  technical  reports  are  usually  submitted  to  the  funding  agency, 
but  are  rarely  posted  on  websites  or  shared  outside  the  agency.  Although  an  email  address  was  set 
up  for  this  study  so  survey  participants  could  send  reports  privately,  relatively  few  technical  reports 
were  identified  for  the  study  considering  the  number  ot  grant  programs  that  require  an  evaluation 
and  arts  organizations  that  conduct  evaluations  for  their  own  purposes. 

The  pool  of  technical  reports  was  further  narrowed  by  the  number  that  did  not  meet  standards  for 
relevance  (87.5%),  often  because  they  were  focused  on  learning  through  the  arts  or  other  related 
areas  such  as  arts  appreciation  or  exposure  to  arts  events.  Of  the  eight  relevant  studies,  only  three 
were  considered  high-quality,  while  the  others  relied  solely  on  anecdotal  evidence  or  pertinent 
information  was  missing  (e.g.,  research  design,  study  population,  methods  ot  data  collection  and 
analysis). 


Page  39 


UUesti  :d  J 


Of  the  three  high-quality  technical  reports,  two  were  generated  from  external  project  evaluations 
conducted  by  nationally  recognized  research  institutions,  and  the  third  was  a national  review  of 
large-scale  assessment  efforts  across  the  country.  All  three  projects  included  the  four  primary  art 
forms  (dance,  music,  theater,  visual  arts)  and  one  project  also  included  literary,  media,  and  folk  arts. 

Of  the  two  high-quality  technical  reports  based  on  evaluations,  the  projects  evaluated  were  very 
different.  One  project  provided  professional  development  and  mentoring  to  classroom  teachers  to 
integrate  the  arts  into  instruction  while  the  other  project  funded  arts  organizations  to  provide  arts 
instruction  directly  to  students  during  the  summer.  Both  projects  examined  student  learning  in  the 
arts,  but  one  focused  on  learning  in  only  visual  arts  while  the  other  assessed  learning  in  the  art 
form(s)  addressed  by  the  projects  funded.  Only  the  summer  instruction  project  assessed  students’ 
artistic  knowledge  and  skills  gained  through  arts  instruction. 

The  third  report  presented  an  overview  of  different  assessments  being  used  throughout  the  nation. 
It  provided  summaries  of  assessments  used  to  measure  student  knowledge  and  skills  in  the  arts,  but 
did  not  directly  assess  arts  learning.  The  study  examined  three  nationwide  arts  assessments  and  five 
statewide  assessments,  and  summarized  the  strengths  and  weaknesses  of  each.  The  following 
provides  a more  detailed  view  of  this  work. 


Example  of  a High-Quality  Technical  Report 


This  technical  report  created  by  a prominent,  nationally  recognized  research  organization 
provides  a review  of  large-scale  assessments  and  practice  in  statewide  K-12  arts  assessment, 
The  document  begins  with  a brief  overview  of  the  history  of  assessment  in  arts  education, 
particularly  with  regard  to  standards  and  accountability.  Next,  the  authors  review  three 
nationally  available  assessment  tools  and  provide  commentarv  on  the  strengths  and 
limitations  of  each.  In  addition,  the  report  includes  a discussion  of  the  various  approaches 
taken  by  five  states  that  include  the  arts  as  part  of  their  standardized  student  achievement 
testing.  The  document  includes  information  regarding  the  technical  quality,  feasibility,  and 
fairness  of  the  various  large-scale  educational  assessments,  and  concludes  with  a section 
discussing  lessons  learned  and  how  each  can  contribute  to  an  effective  arts  assessment  and 
accountability’  system.  Of  particular  interest  to  the  authors  is  creating  a balanced 
assessment  system,  which  combines  comprehensiveness,  coherence,  and  continuity. 


Several  key  findings  emerged  from  the  review  of  technical  reports.  First,  few  research  and 
evaluation  reports  are  made  public  or  easily  available.  Second,  many  more  of  the  publiclv  available 
studies  focus  on  learning  through  the  arts  and  the  benefits  of  arts  instruction  on  other  core  content 
areas  than  focus  on  learning  in  the  arts  and  knowledge  and  skills  specific  to  the  arts.  Lastly,  it  is 
important  to  identify  and  work  with  a quality  evaluator,  as  not  all  people  or  agencies  claiming  to 
conduct  evaluations  have  the  full  set  of  skills  necessary  to  conduct  a high-quality  evaluation.  The 
following  demonstrates  how  working  with  an  external  evaluator  mav  not  always  result  in  a high- 
quality’  technical  report. 
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Example  of  a Low-Quality  Technical  Report 


This  evaluation  report  was  created  by  an  external  evaluator  for  a local  arts  organization  that 
received  a federal  grant  to  implement  an  arts-related  professional  development  program  for 
classroom  teachers.  Although  one  goal  of  the  project  was  to  improve  student  critical 
thinking  in  the  arts  — including  students’  ability  to  understand,  analyze,  discuss,  and  create 
works  across  several  art  forms  — the  project  did  not  use  an  objective  assessment  tool  to 
measure  student  knowledge  or  skills  in  the  arts.  Nearly  all  data  collection  related  to  the 
evaluation  of  this  goal  was  qualitative;  therefore,  the  evidence  of  improvement  in  this 
dimension  was  anecdotal.  Further,  student  surveys  involved  more  affective  domain 
questions  such  enjoyment  of  classes,  being  comfortable  at  school,  and  wanting  teachers  to 
add  more  arts  lessons.  No  objective  measures  of  student  knowledge  or  skills  were 
developed  or  administered,  and  while  qualitative  data  can  be  important  for  providing 
contextual  information  for  framing  findings,  it  is  not  sufficient  as  a measure  of  students’ 
abilities  in  the  arts. 


Technical  reports  from  evaluations  and  research  studies  can  provide  valuable  information  about 
effective  practices,  successful  methodologies,  and  program  outcomes.  While  evaluation  is  often  a 
requirement  for  funding,  very  few  evaluation  reports  were  publicly  available.  Further,  many  of  the 
research  reports  identified  were  focused  on  learning  through  the  arts  than  in  the  arts.  Only  three 
technical  reports  were  considered  both  relevant  to  the  study  and  high-quality.  Wider  dissemination 
of  such  reports  could  provide  models  of  successful  programs  for  others  to  replicate  and  establish 
common  practices  for  research  and  evaluation  in  the  arts. 


SUMMARY  OF  LITERATURE  REVIEW 


Several  themes  emerged  from  the  review  of  the  literature  relevant  to  assessment  of  student 
knowledge  and  skills  in  the  arts.  Key  findings  from  the  literature  review  and  subsequent  content 
analysis  included: 

Much  Information  is  Available,  but  Items  Meeting  Minimum  Standards  of  Quality  and 
Relevance  are  in  Short  Supply  — The  research  team  identified  and  reviewed  over  1 ,000 
documents,  with  727  separate  items  analyzed  and  included  in  a Master  Tracking  database.  More 
than  half  did  not  meet  the  standards  for  relevance,  including  research  studies  and  evaluation  reports 
focused  on  learning  through  the  arts  rather  than  in  the  arts  and  arts-related  assessments  that  measured 
components  outside  of  student  knowledge  or  skills,  such  as  appreciation,  participation,  or 
satisfaction.  In  addition,  literature  was  excluded  from  the  study  for  not  meeting  minimum  quality 
requirements.  While  the  proportion  of  literature  that  met  the  standards  lor  both  relevance  and 
quality  varied  by  document  type,  overall  only  one-fifth  of  the  individual  documents  identified  met 
the  requirements  for  both  relevance  and  quality.  This  finding  points  to  an  overall  lack  of  high- 
quality  tools  and  information  directly  related  to  assessing  student  knowledge  and  skills  in  the  arts. 

Research,  Evaluation,  and  Technical  Reports  are  Not  Widely  Available  - Although  arts- 
related  assessment  is  more  common  than  ever,  very  few  research  and  evaluation  reports  were 
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publicly  available.  Evaluation  is  often  a requirement  of  arts-related  federal  and  foundation  grants, 
yet  the  resulting  reports  are  generally  submitted  to  the  funder  and  never  disseminated  further.  ( )f 
the  technical  reports  publicly  available,  the  vast  majority  address  learning  through  the  arts  and  the 
benefits  of  arts  instruction  on  student  achievement  in  other  core  content  areas  rather  than  student 
knowledge  and  skills  in  the  arts.  The  ability  to  access  evaluation  and  technical  reports  previously 
disseminated  would  be  useful  to  arts  organizations  and  evaluators,  allowing  them  to  learn  what 
methodologies  have  been  used  in  the  past  and  with  what  results,  identify  and  build  upon  best 
practices,  and  avoid  pitfalls  through  lessons  learned. 

Lack  of  a Single,  Comprehensive  Clearinghouse  for  Tools,  Information,  and  Resources 
Focused  on  Assessing  Student  Knowledge  and  Skills  In  the  Arts  - The  items  identified 
through  the  literature  review  process  were  scattered  across  many  websites,  journal  articles,  books, 
and  other  documents.  Often,  the  relevant  and  high-quality  literature  was  mixed  in  with  irrelevant 
and  lesser-quality’  materials.  No  single  source  emerged  as  providing  relevant  information,  offering 
high-quality  tools,  presenting  effective  models  of  assessment,  or  guiding  users  through  the' 
assessment  process.  A single  source  or  clearinghouse  offering  users  the  tools,  information,  and 
resources  needed  to  conduct  valid  and  reliable  assessments  of  student  learning  in  the  arts  would  be 
beneficial  to  the  field. 

Need  for  Professional  Development  Regarding  Rubrics  - One  key  finding  of  the  literature 
review  was  the  need  for  professional  development  regarding  rubrics.  While  the  literature  review 
located  a large  number  of  tools  identified  as  rubrics,  the  majority  were  more  aligned  with  simple 
checklists,  rating  sheets,  or  things  to  look  for  during  a performance.  In  addition,  many  of  the  tools 
classified  as  rubrics  were  poorly  constructed,  lacking  both  clear  criteria  and  quality  gradations  for 
meaningful  scoring.  As  a whole  they  were  too  subjective  and  overly  general,  resulting  in  a lack  of 
validity7  for  assessing  student  skills  in  the  arts.  This  points  to  a need  in  the  field  for  professional 
development  as  to  what  constitutes  a rubric,  how  it  is  properly  used,  and  what  components  are 
necessary  to  develop  or  select  a high-quality  rubric. 
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Chapter  4: 
Survey  Findings 
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Arts  Assessment  from  the  Perspective  of 
District  and  School  Staff 


District  and  school  staff  comprised  the  majority  of  survey  respondents,  representing  approximately 
two-thirds  of  all  survey  respondents  included  in  the  study.  This  large  pool  of  respondents  provides 
insight  into  art  assessment  practices  in  more  formal  K-12  education  settings,  where  the  majority  of 
students  obtain  their  arts  experiences.  This  section  of  the  chapter  presents  findings  tor  district  anti 
school  staff.  It  begins  with  a description  of  the  respondents,  including  their  primary  role,  grade 
levels  served,  and  the  types  of  arts  experiences  they  provide.  This  is  followed  by  findings  related  to 
their  experiences  with  assessing  students’  arts  knowledge  and  arts  skills.  This  section  concludes  with 
a discussion  of  the  needs  of  the  field  related  to  arts  assessment  from  the  district  and  school 
perspective. 


RESPONDENT  CHARACTERISTICS 


One  objective  of  the  survey  was  to  capture  basic  information  about  respondents  to  better 
understand  their  role,  the  types  of  arts-related  services  they  provide,  and  their  experiences  with  the 
assessment  of  student  knowledge  and  skills  in  the  arts.  For  district-  and  school-level  stall,  this 
included  collecting  more  specific  details  about  their  primary  role  within  their  job  position.  Of  the  90 
respondents  who  identified  themselves  as  district  staff,  the  vast  majority'  (88.9%)  indicated  they 
served  as  an  arts  coordinator,  liaison,  or  department  chair.  Accounting  for  only  2.2%  of  the  district 
staff,  two  respondents  reported  they  were  district  superintendents  or  assistant  superintendents.  The 
remaining  district  staff  (8.9%)  had  other  responsibilities  in  their  district  such  as  the  administrator  of 
after-school  programs,  grants  administrator,  or  project  coordinator  of  a districtwide  arts  program 
(Exhibit  13). 


Exhibit  13  — Primary  Role  of  District  Staff 


District  Staff 

n 

% 

District  arts  coordinator/liaison/chair 

80 

88.9 

Other  district  staff 

8 

8.9 

District  super  or  assistant  superintendent 

2 

2.2 

District  staff  n = 90 


Of  the  2,079  survey  respondents  who  indicated  they  were  school  staff,  about  two-thirds  (63.0%) 
identified  themselves  as  arts  educators  or  specialists,  which  would  include  specialized  teachers  who 
provide  arts  instruction  at  a single  school  and  those  who  provide  arts  instruction  across  multiple 
schools.  Slightly  more  than  one-third  (35.6%)  reported  they  were  classroom  teachers,  and  only 
seven  (0.3%)  reported  their  primary  role  was  school  principal.  Twenty-two  school  staff  respondents 
(1.1%)  had  other  responsibilities  such  as  special  education  instructional  assistant,  librarian,  theater 
coach,  and  after-school  arts  teacher  (Exhibit  14). 
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Exhibit  14  - Primary  Role  of  School  Staff 


School  Staff 

n 

% 

Arts  educator/arts  specialist 

1,309 

63.0 

Classroom  teacher 

741 

35.6 

Other  school  staff 

22 

1.1 

Principal 

7 

0.3 

School  staff  n — 2,079 


The  survey  also  inquired  about  the  grade  levels  served;  district  staff  were  asked  to  respond  for  their 
entire  district  and  school  staff  were  asked  to  respond  just  for  their  school.  Respondents  were  asked 
to  check  all  the  grade-level  ranges  that  applied  to  their  district  or  school  (e.g.,  elementary,  middle, 
and  high).  Not  surprisingly,  the  majority  of  district  staff  reported  their  districts  served  all  three 
grade-level  ranges,  with  95.1%  indicating  their  districts  served  elementary  school  grades,  90.2% 
indicating  their  district  served  middle  school  grades,  and  89.0%  indicating  their  district  served  high 
school  grades.  Among  the  1,922  school  staff  respondents,  about  two-thirds  (64.6%)  worked  in 
schools  that  served  high  school  grades,  about  one-third  worked  in  schools  that  served  elementary 
school  grades  (32.9%),  and  29.2%  worked  in  schools  that  served  middle  school  grades  (Exhibit  15). 


Exhibit  15  - Grades  Served  by  Job  Position 


District  Staff  School  Staff 


n 

% 

n 

% 

Elementary  school  grades 

78 

95.1 

632 

32.9 

Middle  school  grades 

74 

90.2 

561 

29.2 

High  school  grades 

73 

89.0 

1,242 

64.6 

District  staff  n = 82,  school  staff  n = 1,922 


In  addition  to  their  primary  role  and  grade  levels  served,  respondents  were  also  asked  to  indicate 
which  art  forms  were  addressed  by  their  district  or  school.  Again,  district  staff  were  asked  to 
respond  for  their  entire  district  while  school  staff  were  asked  to  respond  solely  for  their  school,  and 
respondents  were  able  to  check  all  art  forms  that  applied  to  their  district  or  school.  As  might  be 
expected,  visual  arts,  music,  and  theater  were  the  most  commonly  reported  art  forms  addressed  by 
both  the  district  and  school  staff  respondents.  Specifically,  94.3%  of  the  district  stall  indicated  their 
districts  offered  visual  arts  and  music,  and  87.5%  indicated  their  district  offered  theater  classes. 
Similarly,  91.8%  of  the  school  staff  indicated  their  schools  offered  visual  arts,  83.3%  reported  their 
schools  offered  music,  and  65.7%  reported  their  school  offered  theater  classes.  Consistent  with 
their  roles  at  the  district,  a higher  percentage  of  district  staff  reported  offering  additional  art  forms 
compared  to  school  staff  (Exhibit  16). 
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Exhibit  16  — Art  Forms  Addressed  by  the  Districts  and  Schools 


District  Staff  School  Staff 


n 

% 

n 

% 

Visual  arts 

83 

94.3 

1,853 

91.8 

Music 

83 

94.3 

1,681 

83.3 

Theater 

77 

87.5 

1,326 

65.7 

Dance 

66 

75.0 

694 

34.4 

Media  arts 

64 

72.7 

908 

45.0 

Musical  theater 

61 

69.3 

946 

46.9 

Literary  arts 

49 

55.7 

883 

43.7 

Folk  arts 

14 

15.9 

153 

7.6 

Opera 

11 

12.5 

73 

3.6 

Other  art  forms'^ 

3 

3.4 

13 

0.6 

District  staff  n = 88,  school  staff  n — 2,019 


WestEd  conducted  statistical  analyses  to  determine  it  respondents  varied  in  terms  of  their 
experiences  with  and  uses  ot  arts  assessments  based  on  the  types  of  art  forms  they  addressed. 
Overall,  responses  based  on  the  type  of  art  form  were  similar.  As  such,  this  report  does  not  discuss 
details  in  relation  to  the  various  types  of  art  forms;  however,  tables  with  data  presented  by  art  form 
are  provided  in  Appendix  C. 


ARTS-RELATED  OFFERINGS 


In  addition  to  reporting  on  the  art  forms  that  districts  and  schools  provided,  respondents  also 
identified  the  arts-related  sendees  their  district  or  school  offered.  In-school  arts  instruction  by  a 
certified  arts  educator  or  specialist  was  the  most  common  sendee  reported  by  both  district  and 
school  staff  (88.6%  and  75.7%,  respectively).  For  other  types  of  sendees,  district  staff  were  more 
likely  to  report  their  district  offered  these  sendees  than  school  staff.  For  example,  district  staff  were 
twice  as  likely  as  school  staff  to  indicate  their  district  offered  professional  development  in  the  arts  to 
teachers  (81.8%  vs.  41.7%),  a district-developed  arts  curriculum  (69.3%  vs.  34.3%),  and  arts 
assessments  (61.4%  vs.  28.0%).  Overall,  district  staff  were  more  likely  to  report  the  offering  or 
availability  of  these  arts-related  sendees  (Exhibit  17),  which  is  expected  since  they  are  responding  on 
behalf  of  a district  and  school  staff  were  responding  on  behalf  of  only  their  school. 


V>  Examples  of  “Other  art  forms”  addressed  included  the  culinary  arts,  industrial  arts,  art  history,  puppetry,  and 
architectural  design. 
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Exhibit  17  - Services  on  Arts  the  School/District  Offered 


District  Staff 

School  Staff 

n % 

n % 

In-school  arts  by  a certified  arts  instructor 

78 

88.6 

1,522 

75.7 

Teacher  professional  development  in  the  arts 

72 

81.8 

838 

41.7 

District  developed  arts  curriculum 

61 

69.3 

689 

34.3 

General  arts  instruction 

57 

64.8 

898 

44.7 

Artist  residencies/artists  in  classrooms 

56 

63.6 

437 

21.7 

After-school  arts  instruction 

55 

62.5 

959 

47.7 

Arts  assessments 

54 

61.4 

564 

28.0 

Teacher/school  developed  arts  curriculum 

53 

60.2 

1,303 

64.8 

Arts  integration 

49 

55.7 

481 

23.9 

In-school  arts  instruction  delivered  by  a classroom  teacher 

41 

46.6 

953 

47.4 

Publisher  developed  arts  curriculum 

18 

20.5 

162 

8.1 

Other  arts  programs 

4 

4.5 

42 

2.1 

District  staff  n = 88,  school  staff  n = 2,01 1 


EXPERIENCE  WITH  ASSESSMENT  OF  STUDENT  KNOWLEDGE  IN  THE  ARTS 


One  goal  of  the  survey  was  to  identify  respondents’  experiences  with  the  assessment  of  student 
knowledge  in  the  arts  (see  the  Chapter  2 - Methodology  for  a definition  of  student  knowledge). 
Thus,  the  survey  included  questions  about  specific  types  of  experiences  (such  as  professional 
development  and  training  on  assessment)  as  well  as  their  use  of  knowledge  assessments  (such  as  the 
types  of  assessments  used  and  who  developed  them).  This  section  presents  findings  from  district 
and  school  staff  about  their  experiences  with  the  assessment  of  student  knowledge  in  the  arts. 

Various  types  of  experiences  among  district  and  school  staff  were  examined  with  respect  to  the 
assessment  of  student  knowledge  in  the  arts.  The  vast  majority  of  respondents  indicated  they  had 
some  experience  with  the  assessment  of  student  knowledge  in  the  arts  — only  1.4%  of  district  staff 
and  4.7%  of  school  staff  reported  having  no  such  experiences.  In  general,  district  staff  had  more 
varied  types  of  experiences  with  knowledge  assessments  than  school  staff.  In  terms  of  where 
respondents  received  training  about  assessment  practices,  three-quarters  of  district  staff  (75.4%)  and 
more  than  half  of  school  staff  (57.1%)  reported  attending  professional  development  workshops 
related  to  the  assessment  of  student  learning,  while  just  under  half  of  both  groups  (44.9%  and 
49.2%,  respectively)  reported  receiving  undergraduate-  or  graduate-level  training  related  to 
assessment.  Consistent  with  their  roles,  district  staff  were  twice  as  likely  as  school  staff  to  indicate 
they  had  developed  knowledge  assessment  tools/ resources  for  use  by  others  (60.9%  and  30.4%, 
respectively)  and  more  than  three  times  more  likely  to  have  conducted  professional  development 
regarding  knowledge  assessments  in  the  arts  (55.1%  vs.  16.1%).  As  shown  in  Exhibit  18,  less  than 
one-quarter  of  school  staff  reported  they  were  involved  in  developing  policies  or  conducting 
research  related  the  assessment  of  student  knowledge  in  the  arts. 
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Exhibit  18  - Experiences  Related  to  the  Assessment  of  Student  Knowledge  in  the  Arts 


District  Staff 

School  Staff 

n 

% 

n 

% 

No  experience  assessing  knowledge  in  the  arts 

1 

1.4 

71 

4.7 

Attended  professional  development 

52 

75.4 

865 

57.1 

Developed  knowledge  assessment  tools/resources 

42 

60.9 

460 

30.4 

Conducted  professional  development 

38 

55.1 

244 

16.1 

Received  undergrad  or  graduate  training 

31 

44.9 

745 

49.2 

Developed  policies  on  knowledge  assessment 

23 

33.3 

309 

20.4 

Conducted  research  on  student  knowledge 

18 

26.1 

254 

16.8 

Required  grantees  to  assess  student  knowledge 

9 

13.0 

29 

1.9 

Other  arts  knowledge  experience 

1 

1.4 

11 

0.7 

District  staff  n — 69,  school  staff  n — 1,514 


In  addition  to  reporting  on  their  various  types  of  experiences  related  to  arts  knowledge  assessment, 
respondents  also  answered  survey  questions  about  their  use  of  knowledge  assessments  in  the  arts. 
First,  they  identified  the  types  of  assessments  they  had  used  to  measure  student  knowledge  in  the 
arts.  The  two  most  commonly  reported  types  of  assessments  used  were  tests  developed  by  a 
classroom  teacher/ teaching  artist  (63.6%  district  and  70.2%  school)  and  non-paper/pencil 
assessments,  which  included  art  projects,  discussions,  oral  presentations,  music  and  theater 
performances,  and  portfolios.  In  general,  performances  and  portfolios  are  more  associated  with 
measuring  skills  than  knowledge,  so  the  high  percentage  of  both  district  and  school  staff  indicating 
they  used  these  types  of  assessments  to  measure  student  knowledge  may  be  an  indication  that  survey 
respondents  do  not  clearly  understand  the  distinction  between  art  knowledge  and  art  skills,  or  they 
may  be  using  assessment  tools  in  ways  other  than  intended. 

Slightly  less  than  half  of  district  (43.9%)  and  school  (48.4%)  staff  indicated  they  surveyed  the  teacher 
and/or  artist  to  collect  their  perspective  on  students’  knowledge.  District  staff  tended  to  report 
using  tests  developed  by  their  school/organization  (40.9%  vs.  23.2%),  tests  from  their  district 
(25.8%  vs.  8.1%),  and  tests  developed  by  an  evaluator  or  external  consultant  (15.2%  vs.  5.8%)  more 
than  school  staff.  Overall,  few  respondents  reported  using  tests  found  on  the  Internet,  computer- 
based  tests,  or  tests  purchased  from  test  developers  (Exhibit  19). 
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Exhibit  19  — Types  of  Assessments  Used  to  Measure  Student  Knowledge  in  the  Arts 


District  Staff  School  Staff 


n 

% 

n 

% 

Test  developed  by  classroom  teacher/teaching  artist 

42 

63.6 

999 

70.2 

Non-paper/pencil  assessments 

42 

63.6 

753 

52.9 

Teacher/artist  survey  of  student  knowledge 

29 

43.9 

689 

48.4 

Test  developed  by  the  school 

27 

40.9 

330 

23.2 

Test  included  with  textbook  or  lesson 

26 

39.4 

456 

32.0 

Test  from  my  district 

17 

25.8 

115 

8.1 

Test  developed  by  evaluator/consultant 

10 

15.2 

82 

5.8 

Test  from  my  state 

7 

10.6 

85 

6.0 

Test  found  on  Internet 

4 

6.1 

182 

12.8 

Computer-based  testing  program 

3 

4.5 

73 

5.1 

Test  purchased  from  testing  agency 

2 

3.0 

15 

1.1 

Other  knowledge  assessments 

1 

1.5 

53 

3.7 

District  staff  n = 66,  school  staff  n — 1,424 


Second,  survey  respondents  identified  who  developed  the  arts  knowledge  assessments  they  used. 
Over  three-fourths  of  district  (76.8%)  and  school  (80.1%)  staff  used  knowledge  assessments 
developed  by  a classroom  teacher  or  school.  More  than  half  of  district  staff  (53.6%)  indicated  they 
had  administered  knowledge  assessments  that  were  district-developed,  while  the  same  held  true  for 
less  than  one-quarter  of  school  staff  respondents  (23.9%).  Finally,  relatively  small  propordons  of 
respondents  reported  using  externally  developed  or  arts  organization-developed  knowledge 
assessments  (Exhibit  20). 


Exhibit  20  - Use  of  Knowledge  Assessments  Developed  by  Various  Sources 


District  Staff 

School  Staff 

n 

% 

n 

% 

Teacher/school-developed 

53 

76.8 

1,213 

80.1 

District-developed 

37 

53.6 

362 

23.9 

Externally  developed 

15 

21.7 

461 

30.4 

Arts  organization-developed 

9 

13.0 

105 

6.9 

District  staff  n — 69,  school  staff  n — 1,514 


Overall,  district  and  school  staff  reported  many  types  of  experiences  with  the  assessment  of  student 
knowledge  in  the  arts,  with  district  staff  generally  reporting  more  types  of  experiences  than  school 
staff.  Regarding  the  use  of  knowledge  assessments,  both  groups  reported  primarily  using  knowledge 
assessments  developed  by  classroom  teachers  or  school  staff,  and  non-paper/pencil  tests.  In 
general,  district  and  school  staff  were  more  likely  to  use  knowledge  assessment  tests  that  were  locally 
developed  than  those  developed  by  external  agencies. 
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EXPERIENCE  WITH  ASSESSMENT  OF  STUDENT  SKILLS  IN  THE  ARTS 


Nearly  all  respondents  indicated  they  had  some  experience  with  the  assessment  of  student  skills  in 
the  arts,  with  only  4.5%  of  district  staff  and  3.7%  of  school  staff  indicating  they  had  no  experience 
with  arts-related  skills  assessment.  Similar  to  findings  about  student  knowledge  in  the  arts,  district 
staff  also  had  more  varied  types  of  experiences  related  to  the  assessment  of  student  skills  in  the  arts 
than  did  school  staff.  In  terms  of  training  received  on  assessing  student  skills,  over  three-fourths  of 
district  staff  (76.1%)  indicated  they  attended  professional  development  on  skills  assessments 
whereas  52.6%  of  school  staff  attended  professional  development  on  the  same  topic.  As  was  the 
case  with  assessing  student  knowledge  in  the  arts,  fewer  than  half  of  district  and  school  staff 
received  university-level  training  on  assessing  student  skills.  Roughly  hall  of  district  (58.2%)  and 
school  (44.4%)  staff  had  developed  art  assessment  tools/resources.  In  alignment  with  the 
responsibilities  of  their  positions,  district  staff  (56.7%)  were  much  more  likely  to  have  conducted 
professional  development  on  skills  assessment  than  school  staff  (15.0%).  Finally,  smaller 
percentages  of  respondents  reported  they  had  developed  policies  on  the  assessment  of  skills  or 
conducted  research  on  student  skills  in  the  arts  (Exhibit  21). 


Exhibit  21  — Experiences  Related  to  the  Assessment  of  Student  Skills  in  the  Arts 


District  Staff 

School  Staff 

n 

% 

n 

% 

No  experience  assessing  skills  in  the  arts 

3 

4.5 

54 

3.7 

Attended  professional  development  on  skills  assessment 

51 

76.1 

772 

52.6 

Developed  art  assessment  tools/resources 

39 

58.2 

651 

44.4 

Conducted  professional  development  on  skills  assessment 

38 

56.7 

220 

15.0 

Received  undergrad  or  graduate  training  on  skills  assessment 

32 

47.8 

707 

48.2 

Developed  policies  on  assessment  of  skills 

21 

31.3 

284 

19.4 

Conducted  research  on  student  skills 

13 

19.4 

229 

15.6 

Required  grantees  to  assess  student  skills 

13 

19.4 

24 

1.6 

Other  arts  skills  experience 

1 

1.5 

6 

0.4 

District  staff  n — 67,  school  staff  n = 1 ,467 


In  addition  to  reporting  on  their  various  types  of  experiences  related  to  the  assessment  of  student 
skills  in  the  arts,  respondents  also  answered  survey  questions  about  their  use  of  arts-related  skills 
assessments.  The  types  of  assessments  used  to  measure  student  skills  in  the  arts  were  similar  for 
district  and  school  staff,  and  both  groups  reported  they  used  a variety  of  assessment  tools.  More 
than  three-quarters  of  respondents  indicated  they  had  used  rubrics,  performance-based  assessments, 
observation  protocols,  and  student  self-assessments  to  measure  student  skills  in  the  arts.  In 
addition,  84.4%  of  district  staff  and  60.6%  of  school  staff  reported  they  used  portfolio  reviews 
(Exhibit  22).  Interestingly,  over  half  of  district  (57.8%)  and  school  (61.3%)  staff  indicated  they  used 
paper-pencil  tests  to  measure  student  skills  in  the  arts.  In  general,  paper-pencil  tests  assess  student 
knowledge,  not  skills.  This  finding  also  points  to  the  fact  that  district  and  school  staff  may  not 
clearly  distinguish  between  arts  knowledge  and  arts  skills. 
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Exhibit  22  - Types  of  Assessments  Used  to  Measure  Student  Skills  in  the  Arts 


District  Staff 

School  Staff 

n 

% 

n 

% 

Rubric 

58 

90.6 

1,246 

88.5 

Performance-based  assessment 

57 

89.1 

1,256 

89.2 

Observation  protocol 

55 

85.9 

1,173 

83.3 

Portfolio  review 

54 

84.4 

853 

60.6 

Student  self-assessment 

50 

78.1 

1,147 

81.5 

Paper-pencil  test 

37 

57.8 

863 

61.3 

Checklist 

31 

48.4 

631 

44.8 

Teacher/artist  survey  of  student  skills 

28 

43.8 

481 

34.2 

Computer  software 

11 

17.2 

94 

6.7 

Other  skills  measures 

5 

7.8 

39 

2.8 

District  staff  n = 64,  school  staff  n = 1,408 


Findings  pertaining  to  the  developers  of  the  skills  assessments  used  were  similar  to  those  regarding 
the  developers  of  student  knowledge  assessments  used.  Approximately  three-quarters  of 
respondents  reported  using  skills  assessments  developed  by  a classroom  teacher  or  school.  While 
more  than  half  of  district  staff  (53.7%)  had  used  skill  assessments  that  were  district-developed,  only 
18.8%  of  school  staff  reported  the  same.  Lastly,  similar  to  findings  related  to  the  development  of 
knowledge  assessments,  only  about  ten  percent  of  respondents  reported  using  externally  developed 
or  arts  organization-developed  skills  assessments  (Exhibit  23). 


Exhibit  23  — Use  of  Skills  Assessments  Developed  by  Various  Sources 


District  Staff 

School  Staff 

n 

% 

n 

% 

Teacher/school-developed 

48 

71.6 

1,133 

77.2 

District-developed 

36 

53.7 

276 

18.8 

Externally  developed 

16 

23.9 

304 

20.7 

Arts  organization-developed 

7 

10.4 

76 

5.2 

District  staff  n = 67,  school  staff  n — 1 ,467 


Findings  about  district  and  school  staff  experiences  with  the  assessment  of  student  skills  in  the  arts 
were  similar  to  those  for  the  assessment  of  student  knowledge  in  the  arts.  Overall,  both  district  and 
school  staff  reported  varied  types  of  experiences  related  to  the  assessment  of  student  skills  in  the 
arts,  as  well  as  extensive  use  of  skills  assessments.  Both  groups  used  a variety7  of  skills  assessment 
tools,  including  rubrics,  observations,  and  portfolio  reviews.  Further,  respondents  were  most  likely 
to  use  assessment  tools  developed  by  classroom  teachers  or  school  staff  such  as  teaching  artists  or 
arts  specialists.  Lastly,  findings  suggest  district  and  school  staff  may  not  make  clear  distinctions 
between  student  knowledge  and  skills  in  the  arts. 
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FINDING  AND  DEVELOPING  ARTS-RELATED  ASSESSMENT  TOOLS 


In  order  to  address  the  current  status  of  arts  assessment,  the  survey  also  asked  participants  specific 
questions  about  the  actions  they  take  when  they  need  to  assess  student  knowledge  or  skills  in  the 
arts.  More  specifically,  the  survey  included  questions  about  the  steps  taken  when  an  assessment  is 
needed  and  where  respondents  search  for  existing  assessment  tools,  as  well  as  inquiring  about  the 
single  most  useful  assessment  tool  they  had  used,  including  who  developed  it,  who  scored  it,  and 
how  the  data  were  used. 

Respondents  reported  using  a variety  of  methods  when  they  needed  to  assess  student  learning. 
Nearly  equal  percentages  of  district  (75.4%)  and  school  (76.2%)  staff  reported  they  would  modify 
existing  tools  they  had  used  previously  to  assess  student  knowledge  or  skills  in  the  arts.  In  addition, 
most  school  staff  (81.4%)  and  district  staff  (67.7%)  indicated  they  would  create  a new  tool.  Large 
proportions  of  respondents  also  reported  they  would  search  for  existing  tools,  use  tools  already 
developed  by  them  or  their  district/school,  or  seek  professional  development/ workshops  on  arts 
assessment  (Exhibit  24).  Overall,  the  findings  suggest  that  district  and  school  staff  employ  myriad 
approaches  when  they  need  assessment  tools. 

Exhibit  24  - Actions  Taken  When  Needing  to  Assess  Student  Knowledge  or  Skills 


District  Staff  School  Staff 


n 

% 

n 

% 

Modify  existing  tool  previously  used 

49 

75.4 

1,088 

76.2 

Create  a new  tool 

44 

67.7 

1,163 

81.4 

Search  for  existing  tool 

44 

67.7 

785 

55.0 

Use  tool  already  developed  by  me/my 
district/school 

37 

56.9 

1,026 

71.8 

Seek  PD/workshops  on  assessment 

36 

55.4 

599 

41.9 

Hire  someone  to  develop  one 

3 

4.6 

8 

0.6 

Other 

0 

0.0 

3 

0.2 

District  staff  n = 56,  school  staff  n = 1 ,428 

Respondents  who  reported  they  would  search  for  existing  tools  when  they  needed  to  assess  student 
knowledge  or  skills  in  the  arts  received  a follow-up  quesdon  that  asked  where  specifically  they  would 
look  to  locate  exisdng  tools.  An  overwhelming  majority  of  respondents  (93.6%  school,  83.3% 
district)  stated  they  would  use  an  Internet  search  engine  such  as  Google  or  Yahoo  to  locate  existing 
assessment  tools,  and  roughly  40%  indicated  they  would  visit  specific  websites.  Commonly  reported 
sites  included  the  Incredible  Art  Department’s  website,  the  National  Art  Education  Association’s 
(NAEA)  website,  the  RubiStar  website,  and  the  Kennedy  Center’s  ArtsEdge  website.  Additionally, 
26.2%  of  district  staff  and  19.5%  of  school  staff  reported  contacting  a specific  agency  when  they 
needed  to  locate  an  existing  tool.  The  NAEA,  the  Educational  Theatre  Association  (EdTA),  the 
National  Endowment  for  the  Arts  (NEA),  and  the  Perpich  Center  for  Arts  Education  were  some  of 
the  agencies  respondents  frequently  identified.  Other  methods  cited  by  respondents  to  identify 
existing  assessment  tools  included  contacting  specific  people  and  using  the  library  (Exhibit  25).  The 
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varied  methods  for  locating  assessment  tools  and  the  reliance  on  Internet  search  engines  points  to  a 
need  for  a single  location  or  vetted  clearinghouse  with  assessment  tools,  scales,  or  item  banks  that 
people  can  use  to  make  more  valid  and  reliable  assessments  of  student  learning  in  the  arts. 


Exhibit  25  - Where  Respondents  Look  for  Existing  Assessment  Tools 


District  Staff 

School  Staff 

n % 

n % 

Search  engine 

35 

83.3 

731 

93.6 

Websites 

16 

38.1 

325 

41.6 

Contact  specific  agency 

11 

26.2 

152 

19.5 

Contact  specific  person 

10 

23.8 

142 

18.2 

Library 

5 

11.9 

210 

26.9 

Other 

3 

7.1 

43 

5.5 

District  staff  n — 42,  school  staff  n — 718 


IDENTIFYING  THE  MOST  USEFUL  ASSESSMENT  TOOL 


Survey  respondents  were  also  asked  to  identify  the  single  assessment  tool  they  found  to  be  most 
useful  in  measuring  student  learning  in  the  arts.  For  that  particular  tool,  participants  were  asked  a 
series  of  questions  including  who  developed  the  tool,  what  was  the  primary  purpose  for  using  the 
tool,  who  scored  the  assessment,  and  how  the  resulting  data  were  used. 

Nearly  half  of  district  and  school  respondents  indicated  the  most  useful  assessment  tool  they  have 
used  was  developed  by  a teaching  artist  or  arts  specialist.  Nearly  twenty  percent  of  district  staff 
indicated  the  most  useful  assessment  tool  was  developed  by  the  school  district  (19.6%)  or  a 
classroom  teacher  (17.4%).  In  contrast,  35.9%  of  school  staff  indicated  their  most  useful  tool  was 
developed  by  a classroom  teacher,  often  themselves  since  the  majority  of  school  staff  participating  in 
the  survey  were  teaching  artists  or  classroom  teachers.  Only  a few  respondents  indicated  their  most 
useful  assessment  tool  was  found  on  the  Internet,  included  with  a textbook,  or  developed  by  a 
testing  agency,  state  education  agency,  or  external  consultant  (Exhibit  26). 
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Exhibit  26 

— Who  Developed  the  Assessment  Tool 

District  Staff 

School  Staff  | 

n 

% 

n 

% 

Teaching  artist/art  specialist 

22 

47.8 

419 

47.1 

School  district 

9 

19.6 

39 

4.4 

Classroom  teacher 

8 

17.4 

319 

35.9 

Found  on  the  Internet 

3 

6.5 

22 

2.5 

School/arts  organization 

2 

4.3 

11 

1.2 

Testing  agency 

1 

2.2 

12 

1.3 

Included  with  textbook  or  lesson  plans 

0 

0.0 

29 

3.3 

State  education  agency 

0 

0.0 

27 

3.0 

Evaluator  or  external  consultant 

0 

0.0 

8 

0.9 

Collaboration  with  external  partner 

0 

0.0 

1 

0.1 

Other 

1 

2.2 

2 

0.2 

District  staff  n = 46,  school  staff  n = 889 


Many  of  the  assessment  tools  identified  by  respondents  as  the  most  useful  served  multiple  purposes. 
The  vast  majority  of  district  (80.4%)  and  school  (82.0%)  staff  reported  one  purpose  of  the 
assessment  tool  was  to  measure  student  progress/learning.  Consistent  with  their  roles,  district  staff 
were  more  likely  than  school  staff  to  report  using  the  tool  for  program  improvement  (47.8%  vs. 
29.0%)  and  program  evaluation  (43.5%  vs.  26.4%)  and  less  likely  to  use  it  for  classroom 
tests/student  grades  (43.5%  vs.  61.1%).  Smaller  percentages  of  survey  respondents  reported  using 
the  tool  because  it  was  required  by  a district,  state,  school,  or  funder  (Exhibit  27). 


Exhibit  27  - Purpose  of  the  Assessment  Tool 


District  Staff  School  Staff 


n 

% 

n 

% 

Measure  student  progress/learning 

37 

80.4 

723 

82.0 

Program  improvement 

22 

47.8 

256 

29.0 

Program  evaluation 

20 

43.5 

233 

26.4 

Classroom  test/grade 

20 

43.5 

539 

61.1 

Required  by  district 

10 

21.7 

150 

17.0 

Required  by  state 

6 

13.0 

81 

9.2 

Required  by  school 

2 

4.3 

181 

20.5 

Required  by  funder 

1 

2.2 

6 

0.7 

Other 

4 

8.7 

26 

2.9 

District  staff  n = 46,  school  staff  n = 882 


Over  half  of  district  (53.3%)  and  school  (59.1%)  staff  indicated  that  a classroom  teacher  graded  the 
assessment  tool  that  they  found  to  be  most  useful.  Additionally,  slightly  less  than  half  of 
respondents  reported  that  teaching  artists/arts  specialists  scored  the  assessment.  Roughly  one-third 
of  respondents  indicated  students  graded  their  own  assessments,  which  may  have  been  in  addition 
to  having  a classroom  teacher  or  teaching  artist  score  the  assessment.  A smaller  proportion  of 
district  (17.8%)  and  school  (25.1%)  staff  reported  the  assessment  was  graded  bv  students’  peers. 
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Less  than  ten  percent  of  respondents  reported  that  the  assessment  was  scored  by  a district  scoring 
committee,  school  district,  evaluator/consultant,  or  testing  agency  (Exhibit  28). 


Exhibit  28  — The  Individuals  and  Institutions  that  Scored  the  Assessment 


District  Staff 

School  Staff 

n % 

n % 

Classroom  teacher 

24 

53.3 

521 

59.1 

Teaching  artists/arts  specialists 

21 

46.7 

390 

44.3 

Student  graded  by  him/herself 

14 

31.1 

324 

36.8 

Student  graded  by  peers 

8 

17.8 

221 

25.1 

District  scoring  committee 

4 

8.9 

13 

1.5 

School  district 

3 

6.7 

12 

1.4 

Evaluator/consultant 

2 

4.4 

26 

3.0 

Testing  agency 

2 

4.4 

14 

1.6 

State  education  agency 

0 

0.0 

9 

1.0 

Arts  organization  staff 

0 

0.0 

9 

1.0 

Other 

1 

2.2 

10 

1.1 

District  staff  n = 45,  school  staff  n = 881 


The  data  gathered  from  the  assessment  tool  respondents  identified  as  most  useful  were  used  for 
multiple  purposes.  For  district  staff,  the  most  commonly  reported  use  of  the  assessment  data  was  to 
inform  program/lesson  improvement,  reported  by  71.1%  of  the  respondents.  The  next  two  most 
frequently  reported  uses  of  the  assessment  data  by  district  staff  were  student  grades  (60.0%)  and 
providing  formative  feedback  to  students  (51.1%).  Eighty-three  percent  of  school  staff  reported 
using  assessment  data  for  student  grades,  the  most  frequently  reported  use  for  this  group.  School 
staff  also  used  data  gathered  from  the  assessment  tool  to  provide  formative  feedback  to  students 
(62.0%)  and  for  program/lesson  improvement  (48.9%).  For  both  district  (42.2%)  and  school 
(32.9%)  staff,  program  evaluation  was  a fairly  common  use  for  the  assessment  data.  Additionally, 
roughly  one-quarter  of  district  staff  reported  using  the  assessment  data  for  district  accountability. 
Very  few  respondents  reported  data  were  used  for  research,  reported  to  a funding  agency,  or 
published  in  a journal  or  conference  presentation  (Exhibit  29). 
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Exhibit  29  — How  the  Assessment  Data  were  Used 


District  Staff  School  Staff 


n 

% 

n 

% 

Program/lesson  improvement 

32 

71.1 

430 

48.9 

Student  grade 

27 

60.0 

729 

82.9 

Formative  feedback  to  students 

23 

51.1 

545 

62.0 

Program  evaluation 

19 

42.2 

289 

32.9 

District  accountability 

12 

26.7 

77 

8.8 

School  accountability 

9 

20.0 

144 

16.4 

Contribute  to  research 

4 

8.9 

33 

3.8 

Reported  to  funding  agency 

4 

8.9 

8 

0.9 

Published  in  journal/conference  presentation 

3 

6.7 

9 

1.0 

Unknown 

1 

2.2 

12 

1.4 

Other 

4 

8.9 

20 

2.3 

District  staff  n = 45,  school  staff  n = 879 


Interestingly,  the  majority  of  the  district  (78.3%;)  and  school  (68.9%)  staff  reported  that  they 
experienced  challenges  using  the  assessment  even  though  they  still  considered  it  the  most  useful 
assessment  tool  they  had  used.  Respondents  specified  a wide  range  of  challenges  associated  with 
using  their  favored  assessment  tool,  including  issues  involving  time,  cost,  the  nature  of  the  arts,  and 
student  factors.  The  most  frequently  reported  challenge  was  time,  with  both  district  and  school  staff 
reporting  insufficient  time  to  create,  modify,  administer,  and/or  grade  the  assessment.  In  addition, 
respondents  noted  the  costs  for  supplies  and  training  as  issues.  Respondents  also  cited  challenges 
using  their  favored  tool  in  classes  where  students  have  different  levels  of  artistic  skill,  language 
ability,  and  reading  ability.  Both  district  and  school  staff  reported  facing  the  challenge  of  a lack  of 
student  effort  and  students  not  taking  self-assessment  seriously.  Some  respondents  cited  the 
“inherent  subjective  nature”  of  grading  arts  skills  as  a challenge,  but  high-quality  assessment  tools 
should  offer  objective  ratings  that  are  consistent  across  muldple  raters.  Such  comments  may 
indicate  respondents  are  using  lower-quality  assessment  tools  or  do  not  fully  understand  the  benefits 
of  student  assessment  in  the  arts,  such  as  providing  formative  feedback  and  measuring  growth  over 
time. 


ARTS  ASSESSMENT  NEEDS  AMONG  DISTRICT  AND  SCHOOL  STAFF 


The  survey  included  open-ended  questions  related  to  the  needs  of  the  field  in  terms  of  assessing 
student  knowledge  and  skills  in  the  arts.  Review  of  the  qualitative  coding  and  analysis  of  the 
idendfied  four  primary  categories: 

1.  Guidance 

2.  Trained  Professionals 

3.  Making  the  Case 

4.  Additional  Needs 
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The  following  sections  present  findings  about  arts  assessment  needs  as  reported  by  district  and 
school  staff. 


GUIDANCE 

Respondents  expressed  a need  for  various  forms  of  guidance  related  to  the  assessment  of  student 
learning  in  the  arts.  For  both  district  and  school  staff,  the  two  most  common  types  of  guidance 
requested  were  a clear  framework  (e.g.,  guidelines  linking  assessment  with  standards,  aligning 
curriculum  to  standards  and  assessment,  alignment  across  grades/schools/districts/states)  and 
sample  exemplar  tools  (e.g.,  specific  assessment  tools,  examples,  and  item  banks).  For  example,  one 
teacher  discussed  the  need  for  more  clear  guidelines  about  skills  students  should  learn  at  each  grade 
level: 


Unlike  many  of  my  colleagues,  I would  like  to  see  more  consistent  and  specific  curricula 
developed.  The  [National  Standards  for  Arts  Education]  are  good,  but  they  are  very 
broad.  I think  we  need  a standard  scope  and  sequence  that  targets  more  specific  skills  at 
each  grade  level.  This  would  be  hard  to  do  nationally,  but  some  general  framework 
identifying  a lew  key  SPECIFIC  skills  to  be  obtained  at  each  grade  level  would  make 
assessment  data  much  more  valid,  reliable,  and  consistent  for  all  students. 

Another  teacher  expressed  how  using  consistent  assessment  tools  could  benefit  the  arts: 

We  do  not  have  readily  accessible,  quality  assessment  tools  in  our  state  for  assessing  the 
arts.  This  puts  art  educators  at  a disadvantage  when  trying  to  communicate  student  gains 
in  the  arts  to  all  others  outside  of  arts  education.  Great  student  assessments  would  help 
paint  a clearer  picture  for  supporting  the  arts  in  such  a data-driven  time. 

About  one-fifth  of  respondents  across  both  districts  and  schools  expressed  a need  for  alternative 
assessments  (e.g.,  performance-based  assessments,  authentic  assessments,  portfolio  reviews)  for 
measuring  student  learning  in  the  arts.  In  addition,  13.8%  of  district  staff  and  10.9%  of  school  staff 
mentioned  a need  for  professional  learning  communities  (PECs)  among  teachers  and  other 
stakeholders,  particularly  in  support  of  better  communication,  increased  sharing  and  collaboration, 
and  as  a method  to  get  feedback  and  discuss  common  concerns.  One  teacher  noted: 

Teachers  need  to  be  able  to  collaborate  on  learning  state  and  federal  standards,  and 
assessing  students  in  line  with  that.  Teachers  in  arts  are  still  working  individually,  and  do 
not  have  time  to  discuss  and  process  what  is  best  for  our  students  in  the  area  of 
assessment.  We  need  to  start  at  the  beginning. 

A similar  proportion  of  respondents  requested  models,  especially  examples  of  where  arts  assessment 
works  and  what  that  looks  like.  The  desire  to  share  with  and  learn  from  others  was  a key  theme 
across  multiple  categories  and  respondent  groups.  As  shown  in  Exhibit  30,  much  smaller 
proportions  of  respondents  reported  needs  related  to  specific  resources  (e.g.,  materials  or  places  to 
go  to  learn  about  arts  assessment  and  successful  practices),  formative  assessment  (both  in  terms  of 
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learning  more  about  formative  assessment  and  developing/locating  formative  assessment  tools),  and 
policy  (legislation  regarding  arts  education;  e.g.  public  policy,  educational  policy). 


Exhibit  30  - Guidance 


District  Staff 

School  Staff 

n % 

n % 

Exemplar  tools 

32 

40.0 

417 

24.3 

Framework 

21 

26.3 

660 

38.5 

Alternative  assessments 

17 

21.3 

324 

18.9 

Professional  learning  communities 

11 

13.8 

187 

10.9 

Models 

14 

17.5 

182 

10.6 

Resources 

3 

3.8 

144 

8.4 

Formative  assessment 

5 

6.3 

25 

1.5 

Policy 

0 

0.0 

12 

0.7 

District  staff  n — 80,  school  staff  n — 1715 


TRAINED  PROFESSIONALS 

Respondents  expressed  a need  for  trained  professionals  in  the  area  of  arts  assessment  and  arts 
instruction.  More  than  one-fifth  of  all  district  and  school  respondents  identified  professional 
development  as  a need  in  the  field,  including  training  on  the  importance  of  arts  assessment  and  how 
to  develop  and  identify  quality  assessment  tools.  Comments  included: 

Training  for  those  doing  the  assessments  so  that  there  is  a clear  understanding  of  what  is 
being  assessed  and  how. 

Professional  development  that  deals  specifically  with  the  unique  assessment  issues 
related  to  the  arts. 

I see  a real  need  to  educate  teachers  about  assessment  in  the  arts,  including  the  concept 
that  assessment  does  not  impede  the  creative  process.  True  assessment  moves  the  arts 
into  a realm  of  “real  curriculum,”  not  just  the  touchy-feely  stuff. 

In  addition,  respondents  identified  the  need  for  arts  instruction  and  assessment  to  be  conducted  bv 
highly  qualified  teachers  and  teaching  artists. 

How  can  we  possibly  be  looking  at  assessing  student  learning  in  the  arts,  when  most 
states  do  not  have  teaching  credentials  available  for  ALL  of  the  arts?  Although  the  arts 
are  identified  as  core  academic  subjects  in  NCLB,  most  colleges  do  not  offer  teacher 
prep  programs  in  theater  and  dance,  and  teachers  with  advanced  degrees  in  theater  and 
dance  are  still  being  deprived  of  the  opportunity  to  receive  credentials  in  these  subject 
areas.  It  is  highly  disrespectful  of  these  two  arts  fields  to  think  that  teachers  with  degrees 
in  English  and  PE,  having  taken  maybe  only  one  course  in  theater  or  dance,  would  be 
able  to  adequately  assess  students. 
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Many  teachers  in  our  area  assess  for  effort  and  not  mastery.  They  do  this  because  they 
are  not  certified  themselves  and  do  not  know  what  to  look  for.  Qualified  teachers  are 
imperative. 

Better  training  for  teachers.  I think  many  teachers  enter  the  field  out  of  just  a love  for 
the  arts,  but  not  necessarily  with  a legitimate  training  for  it.  More  training  equals  better 
opportunities  to  assess  students. 

Theater  and  arts  instructors  need  to  be  taught  at  the  college  level  how  to  assess.  I’ve  had 
many  student  teachers  and  this  is  the  biggest  thing  I have  to  work  with  them  on.  They 
need  to  break  down  performances  into  concrete  elements  that  can  be  assessed  and 
graded  so  that  arts  can  be  seen  as  academic. 

Exhibit  31  presents  data  related  to  the  need  tor  trained  professionals  for  providing  and  assessing 
student  learning  in  the  arts. 


Exhibit  31  - Trained  Professionals 


District  Staff 

School  Staff 

n % 

n % 

Professional  development 

21 

26.3 

357 

20.8 

Certified  arts  educators 

6 

7.5 

107 

6.2 

University  training 

4 

5.0 

23 

1.3 

District  staff  n = 80,  school  staff  n = 1715 


MAKING  THE  CASE 

District  and  school  staff  also  identified  a need  to  “make  the  case”  for  arts  instruction  and 
assessment,  particularly  expressing  the  value  of  arts  instruction  and  using  empirical  research  as  a way 
to  justify  and  legitimize  the  arts  as  an  academic  discipline.  Both  district  (10.0%)  and  school  (1 1.4%) 
staff  identified  valuing  the  arts  as  a need  of  the  field,  including  demonstrating  the  importance  of  the 
arts  and  making  the  arts  as  valued  as  other  content  areas  within  the  school  setting.  Research  was 
also  identified  as  a primary  need  of  the  field,  particularly  as  it  relates  to  providing  defensible 
arguments  in  support  of  the  arts  as  an  academic  discipline.  Respondents  noted: 

When  grants  are  made  available,  we  are  not  considered  for  funding  because  the  school 
improvement  dollars  are  allocated  for  the  language  arts  and  math.  We  are  in  need  of 
data  from  the  assessments  that  can  become  a part  of  the  “data  dashboard”  even  though 
we  are  a part  of  the  core  curriculum.  We  need  a format  for  collecting  the  information 
that  is  consistent  and  aligned  to  our  field  of  study. 

[We  need]  tangible  results  to  be  shared  with  administration  as  to  the  impact  of  the  arts 
on  our  students,  and  why  they  need  to  continue  to  support  these  programs  through 
appropriate  funding. 


Page  59 


UUestEd  J 


Teachers  also  mentioned  the  need  for  advanced  placement  tests  in  dance  and  theater  to  give  them 
more  balanced  standing  with  music  and  visual  arts.  Interestingly,  school  staff  were  much  more  likelv 
than  district  staff  (7.8%  vs.  1.3%)  to  advocate  for  statewide  testing  and  high-stakes  tesdng  in  the 
arts.  One  teacher  expressed  this  need  by  saying: 

Standardized  tests  would  help  - that  seems  to  be  the  only  way  to  convince  people  that 
the  arts  matter. 

Not  surprisingly,  the  second  most  common  need  expressed  by  school  staff  in  this  category  was 
support  from  leadership,  particularly  administrators.  In  the  words  of  one  teacher: 

More  support  from  the  administration  and  district.  They  need  to  believe  in  the  value  of 
arts  in  education  and  stand  behind  the  arts  as  a discipline. 

As  might  be  expected,  no  district  respondents  identified  support  from  leadership  as  a need  of  the 
field  (Exhibit  32). 


Exhibit  32  — Making  the  Case 

District  Staff 

School  Staff 

n 

% 

n 

% 

Value  of  arts 

8 

10.0 

195 

11.4 

Research 

7 

8.8 

92 

5.4 

Statewide/high-stakes  testing 

1 

1.3 

133 

7.8 

Support 

0 

0.0 

176 

10.3 

Data  management 

0 

0.0 

12 

0.7 

Advocacy 

0 

0.0 

1 

0.1 

District  staff  n = 80,  school  staff  n = 1715 


ADDITIONAL  NEEDS  OF  THE  FIELD 

Several  additional  needs  related  to  arts  assessment  were  identified  through  the  survey.  The  most 
commonly  expressed  need,  not  surprisingly,  was  funding  — for  assistance  from  assessment 
professionals,  for  training  and  professional  development  related  to  arts  assessment,  and  for  staff 
time  to  work  on  developing  assessment  tools,  to  name  a few.  Time  is  also  in  short  supply  according 
to  school  staff;  7.8%  identified  more  time  to  pursue  student  assessment  in  the  arts  as  a need  of  the 
field.  One  teacher  commented: 

Limited  time  with  students  prevents  many  teachers  from  adequately  assessing  or 
addressing  individual  students’  knowledge  and  skills.  Therefore,  arts  are  often  not  given 
the  same  weight  as  other  classroom  subjects. 

Some  respondents  spoke  to  the  “subjective  myth”  — that  the  arts  cannot  be  objectively  assessed  — as 
well  as  the  need  to  overcome  that  myth.  Others  advanced  an  overall  anti-assessment  perspective, 
calling  for  less  assessment  in  the  arts  and  implying  that  assessment  practices  take  away  from  the 
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creative  nature  of  the  arts.  While  the  proportion  of  respondents  expressing  an  anti-assessment 
perspective  was  small,  those  respondents  had  a great  deal  to  say  on  the  topic.  Comments  included: 

1 do  not  think  children  should  be  tested  at  all  in  art  classes.  My  students  are  exposed  to 
many  different  cultures,  artists,  art  styles,  folk  arts,  textiles,  painting,  drawing,  etc.  I 
would  not  like  to  have  to  teach  to  a test.  It  would  stop  creativity  entirely. 

I am  concerned  about  the  possibility  of  emphasizing  testing  in  the  arts,  especially  at  the 
elementary  level.  Bad  idea. 

Most  of  us  don’t  need  “assistance.”  Others  need  to  change  their  perception  of  what  true 
assessment  is.  judging  the  arts  as  if  we  were  math  or  science  is  just  plain  wrong. 

Some  teachers  expressed  concern  over  the  need  for  equal  access  to  the  arts  for  all  students  and 
meeting  the  needs  of  diverse  student  populations,  including  English  language  learners,  economically 
disadvantaged  students,  other  special  populations,  and  students  with  varying  skill  levels  in  the  same 
classroom.  One  teacher  noted: 

Assessment  needs  to  be  diversified  to  meet  the  various  students’  abilities. 

Differentiation  among  the  student  body  has  created  this  need.  For  example,  specific 
assessment  tools  are  needed  for  autistic  students. 

Exhibit  33  presents  data  reflecting  respondents’  additional  areas  of  need  related  to  assessing  student 
learning  in  the  arts. 


Exhibit  33  — Additional  Needs 


District  Staff 

School  Staff 

n % 

n % 

Funding 

5 

6.3 

228 

13.3 

Technology 

3 

3.8 

54 

3.1 

Anti-assessment 

2 

2.5 

53 

3.1 

Subjective  myth 

2 

2.5 

35 

2.0 

None 

2 

2.5 

41 

2.4 

Time 

1 

1.3 

133 

7.8 

Diverse  students 

0 

0.0 

45 

2.6 

District  staff  n = 80,  school  staff  n = 1715 


OVERVIEW  OF  DISTRICT  AND  SCHOOL  STAFF  NEEDS 

In  addition  to  separating  the  qualitative  codes  into  categories,  this  section  provides  an  overview  of 
all  the  needs  of  the  field  expressed  by  district  and  school  staff.  As  noted  above,  the  most  often  cited 
needs  were  related  to  guidance,  specifically  in  terms  of  sample  high-quality  assessment  tools,  a 
framework  for  instruction  and  assessment,  professional  development,  and  models  of  successful 
assessment  practices  in  various  settings  (e.g.,  rural  areas).  Exhibit  34  presents  the  number  of 
respondents  who  identified  the  following  as  needs  of  the  field. 
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Exhibit  34  - Overview  of  District  and  School  Staff  Needs 


District  Staff  School  Staff 


n 

% 

n 

% 

Exemplar  tools 

32 

40.0 

417 

24.3 

Framework 

21 

26.3 

660 

38.5 

Professional  development 

21 

26.3 

357 

20.8 

Alternative  assessments 

17 

21.3 

324 

18.9 

Models 

14 

17.5 

182 

10.6 

Professional  learning  communities 

11 

13.8 

187 

10.9 

Value  of  arts 

8 

10.0 

195 

11.4 

Research 

7 

8.8 

92 

5.4 

Certified  arts  educators 

6 

7.5 

107 

6.2 

Funding 

5 

6.3 

228 

13.3 

Formative  assessment 

5 

6.3 

25 

1.5 

University  training 

4 

5.0 

23 

1.3 

Resources 

3 

3.8 

144 

8.4 

Technology 

3 

3.8 

54 

3.1 

Anti-assessment 

2 

2.5 

53 

3.1 

Subjective  myth 

2 

2.5 

35 

2.0 

None 

2 

2.5 

41 

2.4 

Statewide/high  stakes  testing 

1 

1.3 

133 

7.8 

Time 

1 

1.3 

133 

7.8 

Support 

0 

0.0 

176 

10.3 

Equal  access/diverse  students 

0 

0.0 

45 

2.6 

Policy 

0 

0.0 

12 

0.7 

Data  management 

0 

0.0 

12 

0.7 

Advocacy 

0 

0.0 

1 

0.1 

District  staff  n = 80,  school  staff  n = 1715 
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Arts  Assessment  from  the  Perspective  of 
Policymakers,  Arts  and  Cultural  Organization 
Staff,  and  Researchers 

Arts  education  and  the  assessment  of  learning  also  takes  place  outside  of  the  formal  education 
setting  described  by  the  district  and  school  perspective  above.  This  section  of  the  chapter  mirrors 
the  above  section,  but  presents  findings  from  policymakers,  arts  and  cultural  organization  staff,  and 
researchers.  Findings  for  policymakers  are  reported  separately  for  state  and  county  office  of 
education  staff  and  state  and  county  arts  council  staff.  Data  from  arts  organization  and  cultural 
organization  staff  were  aggregated  and  are  presented  as  one  group,  arts/cultural  organization  staff. 
Data  from  arts  researchers  and  arts  evaluators  were  also  aggregated  and  are  presented  as  one  group, 
arts  researchers/evaluators.  For  more  information  about  why  these  groups  were  aggregated,  see  the 
Methodology  section  of  this  report  (Chapter  2). 

This  section  of  the  chapter  presents  findings  for:  (1)  state  and  county  office  of  education  staff;  (2) 
state  and  county  arts  council  staff;  (3)  arts  and  cultural  organization  staff;  and  (4)  arts  researchers 
and  evaluators.  It  begins  with  a description  of  the  respondents,  including  their  primary  roles  and  the 
types  of  arts  experiences  they  offer.  This  is  followed  by  findings  related  to  their  experiences  with 
assessing  students’  arts  knowledge  and  arts  skills.  This  section  concludes  with  a discussion  of  the 
needs  of  the  field  related  to  arts  assessment  from  the  perspectives  of  policymakers,  arts/ cultural 
organization  staff,  and  arts  researchers. 


RESPONDENT  CHARACTERISTICS 


Survey  respondents  who  indicated  their  job  position  was  something  other  than  district  or  school 
staff  were  asked  a follow-up  question  to  further  define  their  primary  role.  In  many  cases,  the 
primary  role  was  the  same  as  their  job  position,  but  in  some  cases  the  roles  were  more  defined.  For 
example,  among  those  who  answered  their  job  position  was  state/ county7  office  ot  education  staff, 
20.3%  identified  themselves  as  teaching  artists  or  arts  instructors.  Among  arts  and  cultural 
organization  staff,  76.8%  cited  their  primary  role  as  arts  organization  staff,  13.7%  cited  teaching 
artist  or  arts  instructor,  7.9%  cited  cultural  organization  staff,  and  0.5%  cited  evaluator  as  their 
primary  role  within  their  arts  organization.  The  primary  role  among  researchers  also  varied,  with 
42.9%  identifying  themselves  as  institution  of  higher  education  stall,  31.2%  identifying  themselves 
as  evaluators,  and  24.9%  identifying  themselves  as  researchers  (Exhibit  35). 
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Exhibit  35  - Primary  Role  within  Job  Position 


State/County 

State/County 

Arts/Cultural 

Arts 

Office  of 

Office  Arts 

Organization 

Researchers/ 

Education  Staff 

Council  Staff 

Staff 

Evaluators 

n % 

n % 

n % 

n % 

State/county  education  staff 

60 

75.9 

34 

40.5 

1 

0.1 

0 

0.0 

Teaching  artist/art  instructor 

16 

20.3 

1 

1.2 

115 

13.7 

2 

1.0 

Arts  organization  staff 

2 

2.5 

0 

0.0 

645 

76.8 

0 

0.0 

Evaluator 

1 

1.3 

1 

1.2 

4 

0.5 

64 

31.2 

State/county  arts  council  staff 

0 

0.0 

48 

57.1 

0 

0.0 

0 

0.0 

Cultural  organization  staff 

0 

0.0 

0 

0.0 

66 

7.9 

0 

0.0 

Institution  of  higher  education  staff 

0 

0.0 

0 

0.0 

3 

0.4 

88 

42.9 

Researcher 

0 

0.0 

0 

0.0 

3 

0.4 

51 

24.9 

Other 

0 

0.0 

0 

0.0 

3 

0.4 

0 

0.0 

Office  of  education  staff  n = 79,  arts  council  staff  n — 84,  arts/cultural  organization  staff  n — 840, 
researchers/evaluators  n = 205 


The  fact  that  job  positions  earn’  multiple  roles  is  common  practice  in  the  arts  education  field, 
especially  for  smaller  arts  organizations  and  during  hard  economic  times.  For  this  survey, 
respondents  were  limited  to  selecting  one  primary  role  although  many  probably  could  have  checked 
multiple  roles.  This  breakdown  of  primary  roles  within  job  positions  provides  additional  insight  into 
the  survey  respondents,  and  adds  some  contextual  background  for  how  respondents  answered  other 
questions.  For  example,  researchers  who  reported  their  organization  offered  graduate  training  on 
arts  assessment  most  likely  were  employed  by  an  institution  of  higher  education. 

Respondents  were  also  asked  to  identify  the  art  forms  addressed  by  their  agency  or  organization.  In 
general,  state/ county  office  of  education  staff  and  state/ county'  arts  council  staff  reported 
addressing  more  art  forms  than  arts/cultural  organization  staff  and  arts  researchers/evaluators.  This 
was  expected  since  staff  at  the  state  and  county  levels  often  develop  policies  and  programs  that 
apply  to  all  art  forms.  This  was  especially  evident  among  arts  council  staff,  with  more  than  50% 
addressing  each  art  form.  In  contrast,  many  arts  and  cultural  organizations  specialize  in  one  art  form 
as  indicated  by  the  fact  that  no  art  form  was  addressed  by  more  than  hall  the  arts  and  cultural  staff. 
Visual  arts  and  music  were  the  top  two  art  forms  addressed  by  the  respondents’  agencies  and 
organizations,  and  folk  arts  and  opera  were  the  least-addressed  art  forms  (Exhibit  36). 
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Exhibit  36  — Art  Forms  the  Agency/ Organization  Addressed 


State/County 

State/County 

Arts/Cultural 

Arts 

Office  of 

Office  Arts 

Organization 

Researchers/ 

Education  Staff 

Council  Staff 

Staff 

Evaluators 

n % 

n % 

n % 

n % 

Visual  arts 

60 

82.2 

64 

78.0 

337 

41.7 

128 

67.4 

Music 

53 

72.6 

56 

68.3 

379 

46.8 

79 

41.6 

General  arts  education 

49 

67.1 

73 

89.0 

223 

27.6 

98 

51.6 

Theater 

47 

64.4 

55 

67.1 

205 

25.3 

74 

38.9 

Dance 

45 

61.6 

57 

69.5 

250 

30.9 

63 

33.2 

Media  arts 

23 

31.5 

51 

62.2 

136 

16.8 

59 

31.1 

Musical  theater 

19 

26.0 

50 

61.0 

145 

17.9 

43 

22.6 

Literary  arts 

13 

17.8 

54 

65.9 

108 

13.3 

48 

25.3 

Folk  arts 

9 

12.3 

52 

63.4 

82 

10.1 

24 

12.6 

Opera 

8 

11.0 

48 

58.5 

66 

8.2 

18 

9.5 

Other 

0 

0.0 

2 

2.4 

13 

1.6 

3 

1.6 

Office  of  education  staff  n = 73,  arts  council  staff  n = 82,  arts/cultural  organization  staff  n = 809, 
researchers/evaluators  n = 190 


WestEd  conducted  statistical  analyses  to  determine  il  respondents  systematically  varied  in  their 
survey  responses  based  on  the  art  forms  their  agency/organization  addressed.  Overall,  response 
patterns  based  by  the  type  of  art  form  were  similar.  As  such,  this  report  does  not  discuss  details  in 
relation  to  the  various  types  of  art  forms;  however,  tables  with  data  presented  by  art  form  are 
provided  in  Appendix  C. 


ARTS-RELATED  OFFERINGS 


Not  surprisingly,  the  types  of  arts-related  sendees  each  group  of  respondents  offered  were  aligned 
with  their  primary  role.  For  example,  more  than  half  of  state  and  county7  office  of  education  staff 
indicated  they/their  agency  offered  professional  development  for  classroom  teachers  (65.  3%), 
developed  arts  curriculum  (65.3%),  developed  arts-related  policies  (58.3%),  and  offered  professional 
development  for  teaching  artists  (50.0%).  In  comparison,  state  and  county  arts  council  staff 
reported  higher  incidences  of  providing  funding  and  grants  to  arts  organizations  (86.3%),  offering 
professional  development  to  teaching  artists  (80.0%)  and  classroom  teachers  (66.3%),  supporting 
artist  residencies/artists  in  the  classroom  programs  (61.3%),  and  advocating  for  the  arts  (60.0%). 
Exhibit  37  provides  an  overview  of  the  various  sendees  offered  by  the  different  groups  of 
respondents. 
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Exhibit  37  - Services  on  Arts  the  Agency/ Organization  Offered 


State/County  State/County  Arts/Cultural  Arts 

Office  of  Office  Arts  Organization  Researchers/ 

Education  Staff  Council  Staff  Staff  Evaluators 


n 

% 

n 

% 

n 

% 

n 

% 

Classroom  teacher  professional 
development 

47 

65.3 

53 

66.3 

433 

54.0 

107 

57.8 

Develop  arts  curriculum 

47 

65.3 

25 

31.3 

390 

48.6 

89 

48.1 

Develop  arts  policies 

42 

58.3 

41 

51.3 

63 

7.9 

26 

14.1 

Teaching  artist  professional 
development 

36 

50.0 

64 

80.0 

453 

56.5 

99 

53.5 

Classroom-based  arts  instruction 

34 

47.2 

26 

32.5 

463 

57.7 

92 

49.7 

Arts  advocacy 

30 

41.7 

48 

60.0 

282 

35.2 

57 

30.8 

General  arts  instruction 

30 

41.7 

16 

20.0 

297 

37.0 

80 

43.2 

Develop/administer  arts 
assessments 

30 

41.7 

21 

26.3 

166 

20.7 

45 

24.3 

Program  evaluation 

29 

40.3 

38 

47.5 

249 

31.0 

91 

49.2 

Arts  education  research 

22 

30.6 

26 

32.5 

127 

15.8 

106 

57.3 

Pre-service  teacher  training  in  the 
arts 

21 

29.2 

13 

16.3 

174 

21.7 

95 

51.4 

Provide  funding/grants  to  arts  orgs 

19 

26.4 

69 

86.3 

62 

7.7 

9 

4.9 

Summer  arts  programs 

12 

16.7 

18 

22.5 

455 

56.7 

43 

23.2 

Graduate  level  training  in  arts 
education 

10 

13.9 

4 

5.0 

84 

10.5 

84 

45.4 

Artist  mentoring  of  teachers 

10 

13.9 

21 

26.3 

234 

29.2 

36 

19.5 

After  school  arts  programs 

10 

13.9 

20 

25.0 

396 

49.4 

39 

21.1 

After  school  arts  instruction 

9 

12.5 

14 

17.5 

423 

52.7 

33 

17.8 

Artist  residencies  in  classrooms 

6 

8.3 

49 

61.3 

421 

52.5 

34 

18.4 

Private  arts  instruction 

3 

4.2 

4 

5.0 

289 

36.0 

11 

5.9 

Other  arts  programs 

1 

1.4 

8 

10.0 

62 

7.7 

8 

4.3 

Office  of  education  staff  n = 72,  arts  council  staff  n = 80,  arts/cultural  organization  staff  n — 802, 
researchers/ evaluators  n = 1 85 


EXPERIENCE  WITH  ASSESSMENT  OF  STUDENT  KNOWLEDGE  IN  THE  ARTS 


The  survey  included  questions  about  the  various  types  of  experiences  respondents  had  (e.g., 
professional  development),  as  well  as  their  use  of  knowledge  assessments  to  measure  student 
learning.  The  vast  majority  of  all  respondents  indicated  they  had  some  experience  with  the 
assessment  of  student  knowledge  in  the  arts,  with  no  experience  in  assessing  student  learning 
reported  by  15.6%  of  arts/cultural  staff,  13.9%  of  arts  council  staff,  and  fewer  than  10%  of  office  of 
education  staff  and  researchers.  In  terms  of  receiving  training  on  the  assessment  of  student 
knowledge  in  the  arts,  at  least  50%  of  respondents  in  all  four  groups  indicated  they  attended 
professional  development  on  the  topic.  In  contrast,  fewer  than  half  of  respondents  in  all  four 
groups  received  undergraduate-  or  graduate-level  training  on  the  assessment  of  student  knowledge  in 
the  arts,  including  only  23.9%  of  arts/cultural  organization  staff  and  19.4%  of  arts  council  staff. 
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Consistent  with  their  role,  a large  percentage  of  office  of  education  staff  reported  they  conducted 
professional  development  (58.1%)  and  developed  knowledge  assessment  tools/resources  for  use  by 
others  (54.8%).  Arts  council  staff  had  the  highest  percentage  of  respondents  that  reported  they 
required  grantees  to  assess  student  arts  knowledge  (55.6%),  which  is  expected  since  they  were  also 
the  most  likely  group  to  indicate  they  provided  funding  and  grants.  Other  than  their  attendance  at 
professional  development  sessions,  the  arts/cultural  organization  staff  had  relatively  few  types  of 
experiences  with  the  assessment  of  student  knowledge  in  the  arts,  with  less  than  one-third  of 
respondents  reporting  any  other  types  of  experiences.  Finally,  approximately  half  of  the 
researchers/evaluators  indicated  they  conducted  research  on  student  knowledge  (50.3%),  developed 
knowledge  assessment  tools/resources  (49.0%),  and  received  undergraduate  or  graduate  training  on 
the  assessment  of  arts  knowledge  (48.3%).  Exhibit  38  presents  the  types  of  activities  each  group 
experienced  related  to  the  assessment  of  student  knowledge  in  the  arts. 


Exhibit  38  — Experiences  Related  to  the  Assessment  of  Student  Knowledge  in  the  Arts 


State/County 

State/County 

Arts/Cultural 

Arts 

Office  of 

Office  Arts 

Organization 

Researchers/ 

Education  Staff 

Council  Staff 

Staff 

Evaluators 

n % 

n % 

n % 

n % 

No  experience  assessing  knowledge 
in  the  arts 

6 

9.7 

10 

13.9 

84 

15.6 

11 

7.4 

Attended  professional  development 

37 

59.7 

45 

62.5 

270 

50.0 

82 

55.0 

Conducted  professional 
development 

36 

58.1 

22 

30.6 

106 

19.6 

66 

44.3 

Developed  knowledge  assessment 
tools/resources 

34 

54.8 

24 

33.3 

179 

33.1 

73 

49.0 

Received  undergrad  or  graduate 
training 

28 

45.2 

14 

19.4 

129 

23.9 

72 

48.3 

Developed  policies  on  knowledge 
assessment 

24 

38.7 

11 

15.3 

75 

13.9 

38 

25.5 

Conducted  research  on  student 
knowledge 

24 

38.7 

5 

6.9 

91 

16.9 

75 

50.3 

Required  grantees  to  assess  student 
knowledge 

14 

22.6 

40 

55.6 

37 

6.9 

16 

10.7 

Other  arts  knowledge  experience 

2 

3.2 

4 

5.6 

2 

0.4 

6 

4.0 

Office  of  education  staff  n = 62,  arts  council  staff  n = 72,  arts/ cultural  organizadon  staff  n — 540, 
researchers/evaluators  n = 149 

In  addition  to  reporting  on  their  various  types  of  experiences  related  to  arts  knowledge  assessment, 
respondents  also  answered  survey  questions  about  their  use  of  knowledge  assessments  in  the  arts. 
The  types  of  knowledge  assessments  used  by  the  office  of  education  staff,  arts  council  staff, 
arts/cultural  organization  staff,  and  researchers/evaluators  varied  somewhat  across  the  groups.  For 
the  office  of  education  staff,  the  two  most  commonly  used  types  of  assessments  were  tests 
developed  by  a classroom  teacher/ teaching  artist  (60.0%)  and  non-paper/pencil  assessments 
(54.5%).  Examples  of  non-paper/pencil  assessments  reported  by  the  office  of  education  staff  were 
portfolios  and  performance  assessments.  The  arts  council  staff  most  frequently  reported 
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administering  a survey  to  the  teacher/artist  to  assess  student  knowledge  in  the  arts  (62. 1%). 

Similarly,  administering  a survey  to  teachers/artists  was  the  most  commonly  reported  assessment 
type  used  by  arts/cultural  organization  staff  (63.2%)  and  researchers/evaluators  (56.8%).  While  this 
is  one  perspective  of  student  knowledge,  studies  have  not  been  conducted  to  determine  how 
teacher/artist  perceptions  align  with  actual  gains  in  knowledge,  and  often  these  surveys  ask 
respondents  to  provide  one  general  score  for  the  class  as  a whole  rather  than  individual  ratings. 
Nearly  half  of  the  arts  council  staff  (48.3%)  and  researchers/evaluators  (49.2%)  also  employed  non- 
paper/pencil assessments  to  measure  student  knowledge  in  the  arts  (Exhibit  39).  Consistent  with  the 
office  of  education  staff  responses,  the  two  most  frequently  cited  non-paper/pencil  assessments  by 
the  arts  council  staff  and  researchers/evaluators  were  portfolios  and  performance  assessments. 
However,  these  methods  are  better  aligned  with  the  assessment  of  student  skills  than  knowledge, 
likely  indicating  a lack  of  understanding  of  the  distinction  between  art  knowledge  and  art  skills 
similar  to  that  found  among  district  and  school  staff. 


Exhibit  39  - Types  of  Assessments  Used  to  Measure  Student  Knowledge  in  the  Arts 


State/County  State/County  Arts/Cultural  Arts 

Office  of  Office  Arts  Organization  Researchers/ 

Education  Staff  Council  Staff  Staff  Evaluators 


n 

% 

n 

% 

n 

% 

n 

% 

Test  developed  by  classroom 
teacher/teaching  artist 

33 

60.0 

21 

36.2 

135 

30.1 

52 

39.4 

Non-paper/pencil  assessments 

30 

54.5 

28 

48.3 

168 

37.5 

65 

49.2 

Test  developed  by 
school/organization 

21 

38.2 

9 

15.5 

163 

36.4 

33 

25.0 

Teacher/artist  survey  of  student 
knowledge 

20 

36.4 

36 

62.1 

283 

63.2 

75 

56.8 

Test  included  with  textbook  or 
lesson 

16 

29.1 

4 

6.9 

54 

12.1 

22 

16.7 

Test  from  my  district 

14 

25.5 

1 

1.7 

20 

4.5 

6 

4.5 

Computer-based  testing  program 

13 

23.6 

3 

5.2 

16 

3.6 

7 

5.3 

Test  from  my  state 

12 

21.8 

2 

3.4 

34 

7.6 

14 

10.6 

Test  developed  by 
evaluator/consultant 

11 

20.0 

14 

24.1 

112 

25.0 

34 

25.8 

Test  found  on  Internet 

6 

10.9 

3 

5.2 

17 

3.8 

7 

5.3 

Test  purchased  from  testing  agency 

3 

5.5 

0 

0.0 

5 

1.1 

7 

5.3 

Other  knowledge  assessments 

3 

5.5 

12 

20.7 

23 

5.1 

8 

6.1 

Office  of  education  staff  n = 55,  arts  council  staff  n = 58,  arts/ cultural  organization  staff  n = 448, 
researchers /evaluators  n = 132 


Respondents  were  also  asked  to  identify  how  the  knowledge  assessments  they  used  were  developed. 
Sixty'  percent  of  the  office  of  education  staff  reported  they  used  teacher/school-developed 
knowledge  assessments.  In  contrast,  less  than  16%  of  the  arts  council  staff,  arts/cultural 
organization  staff,  and  researchers/evaluators  indicated  they  used  teacher/school-developed 
knowledge  assessments.  Arts  council  staff  (62.5%)  and  arts/cultural  organization  staff  (50.0%)  most 
frequently  reported  they  used  externally'  developed  assessments,  such  as  those  developed  by  testing 
agencies,  consultants,  and  evaluators.  The  researchers/evaluators  also  reported  using  externally 
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developed  (55.0%)  assessments,  often  ones  they  developed  as  the  external  consultant. 
Researchers/evaluators  also  reported  using  district-developed  (48.3%)  and  arts  organization- 
developed  (44.3%)  assessments,  which  are  both  common  practices  in  the  research  field  since  often 
they  are  the  only  assessment  tools  readily  available  (Exhibit  40). 


Exhibit  40  - Use  of  Knowledge  Assessments  Developed  by  Various  Sources 


State/County 

State/County 

Arts/Cultural 

Arts 

Office  of 

Office  Arts 

Organization 

Researchers/ 

Education  Staff 

Council  Staff 

Staff 

Evaluators 

n % 

n % 

n % ~ 

n % 

Teacher/school-developed 

37 

59.7 

10 

13.9 

84 

15.6 

11 

7.4 

District-developed 

24 

38.7 

14 

19.4 

129 

23.9 

72 

48.3 

Externally-developed 

20 

32.3 

45 

62.5 

270 

50.0 

82 

55.0 

Arts  organization-developed 

17 

27.4 

22 

30.6 

106 

19.6 

66 

44.3 

Office  of  education  staff  n — 62,  arts  council  staff  n = 72,  arts/cultural  organizadon  staff  n = 540, 
researchers /evaluators  n = 149 


Overall,  the  four  groups  of  respondents  varied  in  their  experiences  assessing  student  knowledge  in 
the  arts.  A greater  proportion  of  office  of  education  staff  and  researchers/ evaluators  reported 
engaging  in  assessment- related  activities  than  arts  council  and  arts/cultural  organization  staff,  such 
as  creating  assessment  tools,  developing  assessment  policies,  conducting  research,  and  conducting 
professional  development.  Office  of  education  staff  and  researchers/evaluators  also  reported  using 
more  types  of  assessment  tools  to  measure  student  knowledge.  Office  of  education  staff  reported 
using  teacher/ school-developed  assessment  tools  most  often,  while  arts  council  staff,  arts/ cultural 
organization  staff,  and  researchers/ evaluators  were  more  likely  to  use  externally  developed  tools  to 
assess  student  knowledge. 


EXPERIENCE  WITH  ASSESSMENT  OF  STUDENT  SKILLS  IN  THE  ARTS 


The  majority  of  office  of  education  staff,  arts  council  staff,  arts/cultural  organization  staff,  and 
researchers/evaluators  reported  they  had  some  experience  with  the  assessment  of  student  skills  in 
the  arts,  with  fewer  than  twenty  percent  of  each  group  indicating  they  had  no  experience  assessing 
student  arts  skills.  Data  related  to  training  on  the  assessment  of  student  skills  in  the  arts  showed 
that  respondents  from  all  four  groups  were  more  Likely  to  learn  about  skills  assessment  through 
professional  development,  while  between  22.1%  and  43.6%  of  respondents  reported  receiving 
undergraduate-  or  graduate-level  training  on  the  topic. 

Overall,  office  of  education  staff  had  the  most  varied  types  of  experiences  related  to  the  assessment 
of  student  skills  in  the  arts.  I.arge  percentages  of  the  office  of  education  staff  indicated  they 
attended  professional  development  on  skills  assessments  (63.9%),  conducted  professional 
development  on  skills  assessment  (60.7%),  and  developed  art  assessment  tools/resources  (55.7%). 
Half  of  the  arts  council  staff  and  43.2%  of  the  arts/cultural  organization  staff  reported  attending 
professional  development,  while  smaller  percentages  indicated  they  conducted  professional 
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development  on  skills  assessment  (27.9%  and  19.9%)  or  developed  art  assessment  tools/resources 
(27.9%  and  31. 8%).  The  arts  council  staff  stood  out  from  the  other  three  groups  in  that  44.1%  had 
required  grantees  to  assess  student  arts  skills,  while  no  more  than  18%  of  the  other  three  groups 
reported  this  type  of  experience,  which  is  consistent  with  the  respondents’  roles  since  arts  council 
staff  were  much  more  likely  to  offer  arts  funding  and  grants.  Nearly  halt  of  researchers/evaluators 
indicated  they  had  developed  assessment  tools,  conducted  research  on  student  skills  in  the  arts,  and 
received  undergraduate  or  graduate  training  on  skills  assessments  (Exhibit  41). 


Exhibit  41  — Experiences  Related  to  the  Assessment  of  Student  Skills  in  the  Arts 


State/County 

State/County 

Arts/Cultural 

Arts 

Office  of 

Office  Arts 

Organization 

Researchers/ 

Education  Staff 

Council  Staff 

Staff 

Evaluators 

n % 

n % 

n % 

n % 

No  experience  assessing  student 
skills  in  the  arts 

5 

8.2 

13 

19.1 

92 

17.3 

20 

14.3 

Attended  professional  development 
on  skills  assessment 

39 

63.9 

34 

50.0 

230 

43.2 

72 

51.4 

Conducted  professional 
development  on  skills  assessment 

37 

60.7 

19 

27.9 

106 

19.9 

56 

40.0 

Developed  art  assessment 
tools/resources 

34 

55.7 

19 

27.9 

169 

31.8 

65 

46.4 

Received  undergrad  or  graduate 
training  on  skills  assessment 

25 

41.0 

15 

22.1 

126 

23.7 

61 

43.6 

Developed  policies  on  assessment 
of  skills 

24 

39.3 

8 

11.8 

80 

15.0 

35 

25.0 

Conducted  research  on  student 
skills 

18 

29.5 

5 

7.4 

72 

13.5 

64 

45.7 

Required  grantees  to  assess  student 
skills 

11 

18.0 

30 

44.1 

32 

6.0 

15 

10.7 

Other  arts  skills  experience 

0 

0.0 

1 

1.5 

6 

1.1 

0 

0.0 

Office  of  education  staff  n — 61,  arts  council  staff  n = 68,  arts/cultural  organization  staff  n = 532, 
researchers/evaluators  n = 140 

Findings  about  the  types  of  assessments  used  by  the  office  of  education  staff,  arts  council  staff, 
arts/cultural  organization  staff,  and  researchers/evaluators  to  measure  student  skills  indicated  all 
four  groups  use  a wide  variety  of  assessment  methods.  Over  85%  of  office  of  education  staff  and 
over  66%  of  arts  council  staff  and  researchers/evaluators  reported  that  they  used  rubrics,  portfolio 
reviews,  observation  protocols,  performance-based  assessments,  and  student  self-assessments. 
Although  a large  number  of  arts/cultural  organization  staff  reported  they  used  observation 
protocols  (80.9%),  performance-based  assessments  (74.7%),  and  self-assessments  (69.4%), 
comparably  smaller  numbers  indicated  that  they  used  rubrics  (54.3%)  and  portfolio  reviews  (45.7%). 
Additionally,  the  arts  council  staff  (66.0%),  arts/cultural  organization  staff  (68.5%),  and 
researchers/evaluators  (54.7%)  administered  a survey  to  teachers/artists  to  gauge  student  skills  at 
higher  rates  than  did  the  office  of  education  staff  (39.3%).  As  explained  previously,  paper-pencil 
tests  are  generally  not  appropriate  for  measuring  arts  skills.  However,  more  than  one-third  of  all 
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four  groups  reported  using  paper-pencil  tests  to  assess  student  skills  in  the  arts  (Exhibit  42). 
Although  this  is  a smaller  proportion  of  respondents  compared  to  district  and  school  staff,  this 
finding  again  indicates  large  proportions  of  the  field  that  may  not  clearly  distinguish  arts  knowledge 
from  arts  skills,  which  calls  into  question  the  validity  of  some  assessments  used  to  measure  student 
learning  in  the  arts  and  points  to  a need  for  professional  development  on  the  topic.  A common 
understanding  of  the  difference  between  knowledge  and  skills  will  assist  in  communication  across 
groups  and  improve  assessment  practices. 


Exhibit  42  — Types  of  Assessments  Used  to  Measure  Student  Skills  in  the  Arts 


State/County 
Office  of 
Education  Staff 

State/County 
Office  Arts 
Council  Staff 

Arts/Cultural 

Organization 

Staff 

Arts 

Researchers/ 

Evaluators 

n 

% 

n 

% 

n 

% 

n 

% 

Rubric 

53 

94.6 

37 

78.7 

236 

54.3 

98 

83.8 

Portfolio  review 

52 

92.9 

31 

66.0 

199 

45.7 

84 

71.8 

Observation  protocol 

49 

87.5 

37 

78.7 

352 

80.9 

101 

86.3 

Performance-based  assessment 

49 

87.5 

35 

74.5 

325 

74.7 

97 

82.9 

Self-assessment 

48 

85.7 

38 

80.9 

302 

69.4 

88 

75.2 

Checklist 

37 

66.1 

23 

48.9 

114 

26.2 

54 

46.2 

Paper-pencil  test 

31 

55.4 

17 

36.2 

149 

34.3 

51 

43.6 

Teacher/artist  survey  of  student 
skills 

22 

39.3 

31 

66.0 

298 

68.5 

64 

54.7 

Computer  software 

12 

21.4 

1 

2.1 

22 

5.1 

14 

12.0 

Other  skills  measures 

1 

1.8 

2 

4.3 

10 

2.3 

7 

6.0 

Office  of  education  staff  n = 56,  arts  council  staff  n = 47,  arts/cultural  organization  staff  n = 435, 
researchers/evaluators  n = 117 


Consistent  with  the  data  related  to  assessing  student  knowledge  in  the  arts,  63.9%  of  the  office  of 
education  staff  indicated  they  used  teacher/ school-developed  skills  assessments.  I lowever,  for  arts 
council  staff,  arts/cultural  organization  staff,  and  researchers/evaluators,  the  skills  assessments  used 
were  developed  by  more  varied  sources.  While  more  than  50%  of  each  group  used  externally 
developed  knowledge  tests,  this  percentage  dropped  closer  to  25%  for  skills  assessments  (Exhibit 
43). 


Exhibit  43  — Use  of  Skills  Assessments  Developed  by  Various  Sources 


State/County 
Office  of 
Education  Staff 

State/County 
Office  Arts 
Council  Staff 

Arts/Cultural 

Organization 

Staff 

Arts 

Researchers/ 

Evaluators 

n % 

n 

% 

n 

% 

n 

% 

Teacher/school-developed 

39  63.9 

25 

36.8 

172 

32.3 

58 

41.4 

District-developed 

24 

39.3 

6 

8.8 

56 

10.5 

20 

14.3 

Externally  developed 

18 

29.5 

14 

20.6 

145 

27.3 

31 

22.1 

Arts  organization-developed 

18 

29.5 

11 

16.2 

160 

30.1 

25 

17.9 

Office  of  education  staff  n = 61,  arts  council  staff  n = 68,  arts/cultural  organization  staff  n — 532, 
researchers/evaluators  n = 140 
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Overall,  these  staff  indicated  a range  of  experiences  in  assessing  student  skills  in  the  arts.  ( )ffice  of 
education  staff  reported  engaging  in  the  greatest  number  of  activities  related  to  skills  assessment, 
followed  closely  by  arts  council  staff  and  arts  researchers.  Arts/cultural  organization  staff  reported 
the  fewest  types  of  skills  assessment  experiences,  which  is  interesting  since  they  are  more  likely  to  be 
the  providers  of  arts  instrucuon  and  responsible  for  student-level  assessment.  All  four  groups  used 
a variety  of  assessment  tools  to  measure  student  skills,  and  those  tools  were  developed  by  an 
assortment  of  sources. 


FINDING  AND  DEVELOPING  ARTS-RELATED  ASSESSMENT  TOOLS 


In  order  to  address  the  current  status  of  arts  assessment,  the  survey  also  asked  participants  specific 
questions  about  the  actions  they  take  when  they  need  to  assess  student  knowledge  or  skills  in  the 
arts.  More  specifically,  the  survey  included  questions  about  the  steps  taken  when  an  assessment  is 
needed  and  where  respondents  search  for  existing  assessment  tools,  as  well  as  inquiring  about  the 
single  most  useful  assessment  tool  they  had  used,  including  who  developed  it,  who  scored  it,  and 
how  the  data  were  used. 

Survey  findings  indicate  that  all  four  groups  of  respondents  take  multiple  actions  when  they  need  to 
assess  student  knowledge  or  skills  in  the  arts.  Nearly  three-quarters  of  office  of  education  staff 
(73.2%)  and  researchers/evaluators  (72.1%)  and  over  half  of  arts  council  staff  (50.8%)  and 
arts/cultural  organization  staff  (61.5%)  indicated  they  would  create  a new  tool  when  they  had  to 
assess  student  knowledge  or  skills.  Additionally,  58%  to  68%  of  all  four  groups  reported  they  would 
modify  existing  tools  they  previously  used.  In  comparison  to  arts/cultural  organization  staff 
(49.5%)  and  researchers/evaluators  (39.5%),  a larger  proportion  of  the  office  of  education  staff 
(60.7%)  indicated  they  would  seek  professional  development  or  workshops  on  assessment. 
Additionally,  arts  council  staff  were  much  more  likely  to  search  for  an  existing  tool  (80.0%)  and  hire 
someone  to  create  a new  assessment  tool  (44.6%)  than  office  of  education  staff,  arts/cultural 
organization  staff,  and  researchers/evaluators  (Exhibit  44). 


Exhibit  44  - Actions  Taken  When  Needing  to  Assess  Student  Knowledge  or  Skills 


State/County 

State/County 

Arts/Cultural 

Arts 

Office  of 

Office  Arts 

Organization 

Researchers/ 

Education  Staff 

Council  Staff 

Staff 

Evaluators 

n % 

n % 

n % 

n % 

Create  a new  tool 

Modify  existing  tool  previously  used 

41 

35 

73.2 

62.5 

33 

38 

50.8 

58.5 

313 

329 

61.5 

64.6 

93 

87 

72.1 

67.4 

Seek  PD/workshops  on  assessment 

34 

60.7 

35 

53.8 

252 

49.5 

51 

39.5 

Use  one  already  developed  by 
me/my  agency/organization 

31 

55.4 

26 

40.0 

274 

53.8 

72 

55.8 

Search  for  existing  tool 

29 

51.8 

52 

80.0 

302 

59.3 

74 

57.4 

Hire  someone  to  develop  one 

7 

12.5 

29 

44.6 

125 

24.6 

8 

6.2 

Other 

0 

0.0 

3 

4.6 

2 

0.4 

1 

0.8 

Office  of  education  staff  n — 56,  arts  council  staff  n — 65,  arts/ cultural  organization  staff  n = 509, 
researchers/evaluators  n = 129 
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Respondents  who  indicated  they  would  search  for  existing  tools  when  they  need  to  assess  student 
knowledge  or  skills  in  the  arts  were  asked  a follow-up  question  to  identify  where  specifically  they 
would  search.  Respondents  from  each  staff  category  most  frequently  reported  that  they  used  a 
search  engine,  such  as  Google  or  Yahoo,  to  look  for  existing  assessment  tools.  Arts  council  staff 
(64.0%)  were  more  likely  than  the  other  three  groups  of  respondents  to  report  they  would  contact 
specific  agencies.  Across  all  respondents,  agencies  identified  most  often  were  the  National  Guild  for 
Community  Arts  Education,  the  State  Education  Agency  Directors  of  Arts  Education  (SEADAE), 
the  NAEA,  and  the  NEA.  Between  34%  and  42%  of  all  four  groups  of  respondents  indicated  they 
would  visit  websites,  such  as  the  Kennedy  Center’s  ArtsEdge  website  and  the  NEA  website,  when 
they  needed  to  search  for  existing  assessment  tools.  It  should  be  noted  that  the  NEA  website 
currently  does  not  have  any  examples  of  assessment  tools,  but  the  natural  traffic  flow  to  the  website 
lends  itself  to  either  offering  such  examples  or  links  sites  with  examples.  Researchers/evaluators 
were  much  more  likely  than  the  other  three  groups  to  indicate  they  look  for  assessment  tools  at 
libraries  (Exhibit  45),  which  may  stem  from  the  large  percentage  of  researchers/evaluators  who 
work  at  institutions  of  higher  education  and  have  easy  access  to  on-campus  libraries. 


Exhibit  45  — Where  Respondents  Look  for  Assessment  Tools 


State/County  State/County  Arts/Cultural  Arts 

Office  of  Office  Arts  Organization  Researchers/ 

Education  Staff  Council  Staff  Staff  Evaluators 


n 

% 

n 

% 

n 

% 

n 

% 

Search  engine 

22 

75.9 

38 

76.0 

245 

82.2 

58 

79.5 

Contact  specific  agency 

13 

44.8 

32 

64.0 

131 

44.0 

27 

37.0 

Contact  specific  person 

13 

44.8 

22 

44.0 

87 

29.2 

18 

24.7 

Websites 

11 

37.9 

21 

42.0 

102 

34.2 

30 

41.1 

Library 

1 

3.4 

7 

14.0 

57 

19.1 

31 

42.5 

Other 

0 

0.0 

6 

12.0 

4 

1.3 

3 

4.1 

Office  of  education  staff  n = 29,  arts  council  staff  n = 50,  arts/cultural  organization  staff  n = 298, 
researchers/evaluators  n = 73 


IDENTIFYING  THE  MOST  USEFUL  ASSESSMENT  TOOL 


Survey  respondents  were  also  asked  to  identify  the  single  assessment  tool  they  found  to  be  most 
useful  in  measuring  student  learning  in  the  arts.  For  that  assessment  tool,  participants  wTere  asked  a 
series  of  questions  about  the  tool,  including  who  developed  the  tool,  what  was  the  primary  purpose 
for  using  the  tool,  who  scored  the  assessment,  and  how  the  resulting  data  were  used. 

Survey  respondents  reported  a variety'  of  individuals  and  organizations  developed  the  assessment 
tool  they  considered  most  useful.  Approximately  one-third  of  the  office  of  education  staff  reported 
a state  education  agency  (35.9%)  or  a teaching  artist/art  specialist  (33.3%)  developed  the  assessment 
tool  they  found  most  useful.  1 lalf  of  arts  council  staff  reported  a teaching  artist/ art  specialist 
created  their  most  useful  assessment  tool.  Arts/ cultural  organization  staff  and 
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researchers/evaluators  indicated  their  favored  assessment  tools  were  developed  by  varied  sources, 
including  teaching  artists/arts  specialists,  school/arts  organization,  external  consultant/evaluators, 
and  classroom  teachers  (Exhibit  46). 


Exhibit  46  — Who  Developed  the  Assessment  Tool 


State/County 

State/County 

Arts/Cultural 

Arts 

Office  of 

Office  Arts 

Organization 

Researchers/ 

Education  Staff 

Council  Staff 

Staff 

Evaluators 

n % 

n % 

n % 

n % 

State  education  agency 

14 

35.9 

0 

0.0 

7 

2.4 

2 

2.6 

Teaching  artist/art  specialist 

13 

33.3 

7 

50.0 

90 

31.5 

29 

38.2 

Classroom  teacher 

5 

12.8 

0 

0.0 

22 

7.7 

10 

13.2 

School  district 

2 

5.1 

0 

0.0 

3 

1.0 

0 

0.0 

Evaluator  or  external  consultant 

2 

5.1 

3 

21.4 

44 

15.4 

15 

19.7 

School/arts  organization 

1 

2.6 

3 

21.4 

108 

37.8 

10 

13.2 

Collaboration  with  external  partner 

1 

2.6 

1 

7.1 

2 

0.7 

3 

3.9 

Testing  agency 

0 

0.0 

0 

0.0 

4 

1.4 

6 

7.9 

Included  with  textbook  or  lesson 
plans 

0 

0.0 

0 

0.0 

3 

1.0 

0 

0.0 

Found  on  the  Internet 

0 

0.0 

0 

0.0 

2 

0.7 

0 

0.0 

Other 

1 

2.6 

0 

0.0 

1 

0.3 

1 

1.3 

Office  of  education  staff  n — 39,  arts  council  staff  n — 14,  arts/cultural  organization  staff  n = 286, 
researchers/evaluators  n = 76 


The  most  commonly  reported  purpose  for  all  four  groups  of  respondents  was  to  measure  student 
progress/learning.  Nearly  two-thirds  ot  the  arts  council  staff  reported  they  administered 
assessments  for  program  improvement  and  program  evaluation,  and  these  percentages  were  slightlv 
higher  for  arts/cultural  organization  staff.  Additionally,  37.8%  of  the  arts/cultural  organization  staff 
used  the  assessment  tools  due  to  a funder’s  requirement.  Among  researchers  and  evaluators,  60.0% 
employed  the  assessment  tool  for  purposes  of  program  evaluation.  Relatively  low  proportions  of  all 
four  groups  reported  the  assessment  was  required  by  a district  or  school,  or  that  the  assessment  was 
used  for  a classroom  test  or  student  grade  (Exhibit  47). 
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Exhibit  47  - Purpose  of  the  Assessment  Tool 


State/County  State/County  Arts/Cultural  Arts 

Office  of  Office  Arts  Organization  Researchers/ 

Education  Staff  Council  Staff  Staff  Evaluators 


n 

% 

n 

% 

n 

% 

n 

% 

Measure  student  progress/learning 

27 

71.1 

10 

71.4 

216 

76.3 

54 

72.0 

Program  improvement 

13 

34.2 

9 

64.3 

192 

67.8 

35 

46.7 

Program  evaluation 

11 

28.9 

9 

64.3 

207 

73.1 

45 

60.0 

Required  by  state 

10 

26.3 

1 

7.1 

20 

7.1 

10 

13.3 

Required  by  district 

7 

18.4 

1 

7.1 

15 

5.3 

3 

4.0 

Required  by  school 

6 

15.8 

1 

7.1 

27 

9.5 

8 

10.7 

Required  by  funder 

4 

10.5 

4 

28.6 

107 

37.8 

12 

16.0 

Classroom  test/grade 

4 

10.5 

3 

21.4 

38 

13.4 

17 

22.7 

Other 

1 

2.6 

0 

0.0 

18 

6.4 

7 

9.3 

Office  of  education  staff  n — 38,  arts  council  staff  n — 14,  arts/cultural  organization  staff  n = 238, 
researchers/evaluators  n = 75 


The  survey  also  asked  respondents  to  indicate  how  the  assessment  was  scored.  Classroom  teachers 
and  teaching  artists/ arts  specialists  were  commonly  cited  by  all  four  respondent  groups  as  the 
person  who  scored  the  assessment.  Approximately  one-third  of  arts  council  staff  also  indicated 
student  peers  and  arts  organization  staff  were  used  to  score  the  assessment  tool,  and  40.8%  of 
researchers  indicated  the  evaluator  or  external  consultant,  often  themselves,  scored  the  assessment 
(Exhibit  48). 


Exhibit  48  - The  Individuals  and  Organizations  that  Scored  the  Assessments 


State/County  State/County  Arts/Cultural  Arts 

Office  of  Office  Arts  Organization  Researchers/ 

Education  Staff  Council  Staff  Staff  Evaluators 


n 

% 

n 

% 

n 

% 

n 

% 

Classroom  teacher 

17 

43.6 

9 

64.3 

88 

31.7 

28 

36.8 

Teaching  artists/arts  specialists 

14 

35.9 

8 

57.1 

135 

48.6 

32 

42.1 

Student  graded  by  him/herself 

11 

28.2 

3 

21.4 

55 

19.8 

18 

23.7 

District  scoring  committee 

9 

23.1 

0 

0.0 

3 

1.1 

0 

0.0 

Student  graded  by  peers 

7 

17.9 

5 

35.7 

30 

10.8 

13 

17.1 

State  education  agency 

7 

17.9 

0 

0.0 

4 

1.4 

2 

2.6 

School  district 

2 

5.1 

0 

0.0 

2 

0.7 

2 

2.6 

Testing  agency 

5 

12.8 

0 

0.0 

10 

3.6 

3 

3.9 

Arts  organization  staff 

1 

2.6 

5 

35.7 

100 

36.0 

7 

9.2 

Evaluator/consultant 

1 

2.6 

4 

28.6 

74 

26.6 

31 

40.8 

Other 

0 

0.0 

0 

0.0 

11 

4.0 

3 

3.9 

Office  of  education  staff  n = 39,  arts  council  staff  n = 14,  arts/cultural  organization  staff  n — 278, 
researchers/evaluators  n = 76 


Approximately  half  of  office  of  education  staff  reported  the  assessment  data  were  used  for 
formative  feedback  to  students  (56.4%),  program  evaluation  (46.2%),  and  program/lesson 
improvement  (43.6%).  In  contrast,  nearly  all  arts  council  stall  indicated  the  data  were  used  for 
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program  evaluation  (85.7%)  and  program/lesson  improvement  (85.7%).  Arts/cultural  organization 
staff  and  researchers/evaluators  also  reported  high  levels  of  use  tor  program  evaluation  (78.6%  anti 
63.2%)  and  program/lesson  improvement  (72.2%  and  59.2%).  Additionally,  48.4%  of  arts/cultural 
organization  staff  reported  the  data  to  a funding  agency  anti  44.1%  used  assessment  data  to  provide 
formative  feedback  to  students.  Consistent  with  their  roles,  more  researchers/evaluators  used  the 
assessment  data  to  contribute  to  research  (30.3%)  and  to  publish  in  a journal  or  use  lor  a conference 
presentation  (17.1%)  than  any  oi  the  other  three  groups  of  respondents  (Exhibit  49). 


Exhibit  49  - How  Assessment  Data  were  Used 


State/County 

State/County 

Arts/Cultural 

Arts 

Office  of 

Office  Arts 

Organization 

Researchers/ 

Education  Staff 

Council  Staff 

Staff 

Evaluators 

n % 

n % 

n % 

n % 

Formative  feedback  to  students 

22 

56.4 

6 

42.9 

124 

44.1 

32 

42.1 

Program  evaluation 

18 

46.2 

12 

85.7 

221 

78.6 

48 

63.2 

Program/lesson  improvement 

17 

43.6 

12 

85.7 

203 

72.2 

45 

59.2 

Student  grade 

13 

33.3 

2 

14.3 

49 

17.4 

29 

38.2 

District  accountability 

12 

30.8 

0 

0.0 

12 

4.3 

5 

6.6 

School  accountability 

11 

28.2 

2 

14.3 

28 

10.0 

17 

22.4 

Contribute  to  research 

6 

15.4 

4 

28.6 

56 

19.9 

23 

30.3 

Reported  to  funding  agency 

4 

10.3 

7 

50.0 

136 

48.4 

20 

26.3 

Published  in  journal/conference 
presentation 

2 

5.1 

0 

0.0 

12 

4.3 

13 

17.1 

Unknown 

1 

2.6 

0 

0.0 

6 

2.1 

2 

2.6 

Other 

1 

2.6 

0 

0.0 

10 

3.6 

3 

3.9 

Office  of  education  staff  n — 39,  arts  council  staff  n = 14,  arts/cultural  organization  staff  n = 281, 
researchers/evaluators  n = 76 


Similar  to  the  findings  for  district  and  school  staff,  the  majority  of  the  office  of  education  staff 
(69.2%),  arts  council  staff  (64.3%),  arts/cultural  organization  staff  (75.8%),  and 
researchers/evaluators  (71.1%)  reported  they  encountered  challenges  utilizing  the  assessment  they 
found  most  useful.  Again,  the  most  commonly  reported  issue  was  time,  with  all  four  groups 
reporting  they  had  insufficient  time  to  administer  and  score  the  assessment.  Respondents  also  noted 
difficulties  they  encountered  training  artists  to  consistently  administer  and  score  the  assessments. 
Furthermore,  respondents  referred  to  the  subjective  nature  (i.e.,  subjective  myth)  of  arts  assessment 
and  the  difficulties  related  to  quantifying  the  assessment  data.  Finally,  the  respondents  reported  that 
teachers  and  school  administrators’  lack  of  commitment  to  administering  the  assessments  posed  a 
challenge. 


ARTS  ASSESSMENT  NEEDS  AMONG  POLICYMAKERS,  ARTS  AND  CULTURAL  STAFF, 
AND  RESEARCHERS 


Policymakers,  arts  and  cultural  staff,  and  researchers  were  also  asked  about  the  needs  of  the  field  and 
their  organizations  related  to  the  assessment  of  student  learning  in  the  arts.  As  with  district  and 
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school  staff,  qualitative  analysis  of  their  responses  to  these  open-ended  questions  separated  into  four 
primary  categories: 

1.  Guidance 

2.  Trained  Professionals 

3.  Making  the  Case 

4.  Additional  Needs 

GUIDANCE 

Respondents  expressed  a need  for  various  forms  of  guidance  related  to  the  assessment  of  student 
learning  in  the  arts,  including  a clear  framework  linking  standards  to  curriculum  and  assessment, 
exemplar  tools,  alternative  assessments,  models,  resources,  and  professional  learning  communities 
(PLCs).  As  was  the  case  with  district  and  school  staff,  a clear  framework  was  the  most  common 
form  of  guidance  needed  according  to  office  of  education  staff,  arts  council  staff,  and 
researchers/evaluators.  For  arts/cultural  organization  staff,  the  need  for  a framework  was  second 
behind  the  need  for  exemplar  tools.  The  need  for  guidance  in  terms  of  arts  standards,  curriculum, 
and  aligning  both  with  assessment  and  accountability  was  strongly  evident  in  many  responses,  such 
as: 


Broader  discussion  of  arts  assessment  should  include  a balanced  dialogue  about:  1.  What 
we  are  assessing,  2.  Why  we  are  assessing  it,  3.  How  we  are  approaching  assessment, 
and  4.  What  are  the  anticipated  and  unintended  learning  outcomes  we  can  expect  to 
achieve  from  the  assessment. 

[Our  greatest  need  is]  the  development  of  school-,  district-,  and  state-level  accountability 
systems  that  are  based  on  assessing  student  achievement  of  the  standards/benchmarks. 
There  needs  to  be  a state  reporting  process  for  tracking  and  reporting  student 
achievement  in  the  arts. 

A common  vision  and  purpose  for  arts  education  and  how  each  stakeholder  in  the 
delivery  system  meets  those  shared  understandings;  what  their  role  is  within  the  delivery 
system.  A clearer  vision  for  learning  expectations  in  the  arts  that  includes  valid 
measurement  tools  and  how  to  use  assessment  to  inform  student  learning,  curriculum, 
and  instruction.  How  to  use  assessment  data  to  inform  programmatic  decisions. 

More  than  one-quarter  of  respondents  in  each  group  expressed  a need  for  exemplar  tools  for 
assessing  student  learning  in  the  arts.  One  art  organization  respondent  asked  simply  for  “a 
consistent  evaluation  form  that  could  be  used  across  arts  organizations  to  assess  student  learning.” 
Another  called  for  “a  standardized  assessment  tool,  as  creative  as  the  work  we  are  evaluating,  that 
translates  student  work  and  response  into  quantitative  research.”  An  office  of  education  stall 
member  identified  the  need  for  exemplar  tools  as: 
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Quality  arts  assessments  linked  to  national  standards,  based  on  a collaboratively 
developed  content  scope,  ranging  from  unit-embedded  performance  tasks  to  online 
selected  response  items  with  multimedia  prompts. 

The  need  for  alternative  assessments  was  also  a need  expressed  by  all  four  respondent  groups. 
Almost  thirty  percent  of  office  of  education  staff  respondents  identified  alternative  assessments  as  a 
need  of  the  field,  including  one  who  commented: 

Fine  arts  assessments  should  represent  a student’s  body  of  work  over  an  extended  period 
of  time,  not  simply  an  end-of-grading  period  test  of  multiple-choice  questions. 

Assessment  in  fine  arts  should  not  be  about  a “ right/ wrong”  test,  but,  instead,  an 
appraisal  conducted  in  a content-rich  environment  that  is  used  daily  to  evaluate  students’ 
progress,  understanding,  interests,  curiosity,  and  creativity.  We  should  assess  what  we 
value;  not  value  what  we  assess. 

Other  responses  in  support  of  alternative  assessments  included: 

I regret  that  the  arts  have  gone  toward  high-stakes  standards  and  assessments  as  have  the 
humanities  and  sciences.  Instead,  I think  that  the  portfolio  approach,  studio  practice, 
apprenticeships  and  critiques  are  more  in  the  tradition  of  the  arts.  In  particular,  project- 
based  work  that  blends  arts  and  sciences  is  much  needed. 

Standardizing  tests  to  test  a non-standardized  knowledge  is  ineffective.  Students’ 
knowledge  and  skills  should  be  assessed  through  a two-fold  process  of  interview/exam 
and  portfolio  review.  Both  should  address  the  specific  visual  arts  language  being 
addressed  by  the  student  in  relation  to  the  foundational  skills  and  processes  available  to 
them  as  it  relates  to  their  work. 

Many  respondents  were  seeking  arts  assessment  models,  including  systems  of  assessment  or 
evaluation  and  examples  of  where  such  models  are  working.  One  arts  council  representative 
requested  “models  of  different  levels  of  assessment  that  are  affordable  and  applicable  across 
different  types  of  programs.”  An  evaluator  expressed  a desire  to  “preview  arts  assessment  models 
from  successful  school  districts  so  we  can  compare  them  with  our  district  in  order  to  optimize 
[assessment].” 

Resources,  such  as  how-to  materials  and  places  to  go  to  learn  about  arts  assessment,  were 
occasionally  identified  as  needed,  particularly  by  arts  and  cultural  organization  staff  (13.6%)  and 
state/county'  office  of  education  staff  (10.6%).  Comments  regarding  the  need  for  assessment-related 
resources  included: 

As  a smaller  arts  organization,  we  struggle  with  knowing  how  to  develop  surveys,  even 
though  we  use  the  logic  model  of  evaluation.  How  does  a small  group  with  little  staff  or 
time  learn  how  to  design  good  surveys? 

It  would  be  nice  not  to  have  to  spend  hours  looking  for  relevant  assessment  information 
online,  often  coming  away  empty-handed.  II  I had  not  gone  to  recent  National  Arts 
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Education  Association  conferences,  I would  not  have  had  the  resources  to  get  our 
districts  as  far  as  we  have  come  in  setting  up  performance  assessments  in  the  arts. 

Exhibit  50  presents  the  number  of  qualitative  responses  related  to  the  types  of  guidance  needed  for 
assessing  student  learning  in  the  arts. 

O O 


Exhibit  50  - Guidance 


State/County 

State/County 

Arts/Cultural 

Arts 

Office  of 

Office  Arts 

Organization 

Researchers/ 

Education  Staff 

Council  Staff 

Staff 

Evaluators 

n % 

n % 

n % 

n % 

Framework 

26 

39.4 

27 

36.5 

127 

20.6 

61 

36.5 

Alternative  assessments 

19 

28.8 

12 

16.2 

119 

19.3 

40 

24.0 

Exemplar  tools 

17 

25.8 

20 

27.0 

203 

32.8 

45 

26.9 

Models 

10 

15.2 

14 

18.9 

80 

12.9 

27 

16.2 

Resources 

7 

10.6 

7 

9.5 

84 

13.6 

6 

3.6 

Formative  assessment 

6 

9.1 

0 

0.0 

7 

1.1 

3 

1.8 

Professional  learning  communities 

6 

9.1 

7 

9.5 

55 

8.9 

25 

15.0 

Policy 

0 

0.0 

6 

8.1 

7 

1.1 

10 

6.0 

Office  of  education  staff  n = 66,  arts  council  staff  n = 74,  arts/cultural  organization  staff  n = 618, 
researchers/evaluators  n = 167 


TRAINED  PROFESSIONALS 

Respondents  also  expressed  a need  for  trained  professionals  in  the  area  of  arts  assessment, 
encompassing  both  professional  development  and  the  need  for  qualified  arts  educators  to  teach  and 
assess  arts.  Across  the  board,  professional  development,  particularly  training  on  the  importance  of 
quality'  arts  assessment,  was  identified  as  a need  of  the  field.  Each  group  identified  professional 
development  as  a key  need  in  this  category,  ranging  from  a low  of  22.2%  for  researchers/evaluators 
to  a high  of  44.6%  for  arts  council  staff.  In  the  words  of  one  arts  council  respondent: 

[We  need]  better  professional  development,  especially  for  elementary  teachers  who  are 
the  individuals  delivering  arts  instruction.  They  need  continued  follow-up  and 
mentoring  until  assessing  students  becomes  part  of  their  regular  teaching  practice.  They 
assess  reading  and  math  without  thought,  but  assessing  the  arts  is  threatening.  The  field 
needs  to  stop  any  conversation  that  starts  with  “there  are  no  mistakes  in  art”  or  “in  art, 
it's  about  self-expression  and  everything  is  fine.”  It  isn't  fine  and  we've  done  a major 
disservice  to  our  children  and  youth  and  to  teachers  and  parents.  You  CAN  have  muddy 
colors,  your  song  can  be  off  key,  you  may  forget  to  project  your  voice.  There  are 
standards  for  excellence  in  the  arts  and  teachers  need  to  be  aware  of  that.  They  need  to 
know  that  just  as  one  needs  knowledge  and  skills  for  language  arts,  the  arts  has  its  own 
body  of  knowledge  and  skills  that  are  rigorous  and  measurable. 

Additional  comments  regarding  the  importance  of  professional  development  lor  arts  assessment 
included: 
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1 see  a tremendous  lack  of  teacher  education  within  the  public  school  systems  in  general. 
There  seems  to  be  little  or  no  time  for  teacher  training  in  arts  assessment. 

We  need  more  professional  development  than  we  already  have  to  help  teachers  assess 
student  progress  in  very'  short  time  periods.  We  need  to  teach  teachers  how  to  use  the 
revised  curriculum  standards  that  include  more  assessment  assistance. 

[We  need]  professional  development  enabling  arts  educators  and  artists  to  better 
understand  the  benefits  and  techniques  of  assessment  in  their  field. 

Respondents  also  indicated  a need  for  certified  arts  educators  or  teaching  artists,  rather  than  general 
educadon  teachers,  to  be  charged  with  instructing  and  assessing  student  learning  in  the  arts.  Nearly 
ten  percent  of  office  of  education  staff  expressed  this  as  a priority  need  of  the  field,  including  one 
who  offered  the  following  perspective: 

The  arts  involve  both  process  and  final  product,  both  being  valid  to  assess.  But 
assessing  information  and  facts  are  usually  irrelevant,  so  anyone  doing  the  assessing  must 
have  a valid  arts  background  themselves  to  understand  the  context  of  the  assessment. 

Even  advanced  degrees  in  other  education  areas  don’t  help  with  assessing  student 
progress  in  the  arts.  With  all  due  respect,  my  peers  and  people  in  my  industry  are  the 
only  “who”  or  “what”  that  our  organization  could  use.  Everyone  else  is  just  in  the  way. 

Exhibit  51  presents  data  related  to  the  need  for  training  and  trained  professionals  for  assessing 
student  learning  in  the  arts. 


Exhibit  51  - Trained  Professionals 


State/County  State/County  Arts/Cultural  Arts 

Office  of  Office  Arts  Organization  Researchers/ 

Education  Staff  Council  Staff  Staff  Evaluators 


n 

% 

n 

% 

n 

% 

n 

% 

Professional  development 

22 

33.3 

33 

44.6 

142 

23.0 

37 

22.2 

Certified  arts  educators 

6 

9.1 

4 

5.4 

26 

4.2 

3 

1.8 

University  training 

5 

7.6 

4 

5.4 

11 

1.8 

12 

7.2 

Office  of  education  staff  n = 66,  arts  council  staff  n — 74,  arts/cultural  organization  staff  n — 618, 
researchers/evaluators  n = 167 


MAKING  THE  CASE 

Respondents  across  all  four  groups  also  identified  a need  to  “make  the  case”  for  arts  instruction  and 
assessment,  particularly  by  using  empirical  research  and  expressing  the  value  of  the  arts.  More  than 
one-fifth  of  arts/cultural  organization  staff  (24.1%),  arts  researchers/evaluators  (23.4%),  and  arts 
council  staff  (20.3%)  viewed  empirical  research  as  a key  need  in  the  field  of  arts  assessment.  Ideas 
for  necessary’  research  were  varied  and  included  measuring  the  impact  of  arts  instruction  on  other 
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academic  subject  areas,  quantifying  the  effect  of  arts-related  programs,  and  making  the  case  for 
continued  funding  of  both  arts  instruction  and  arts  assessment.  Respondent  comments  included: 

1 would  like  to  see  more  definitive  research  that  supports  the  benefits  of  studying  the 
arts  so  that  the  arts  will  be  recognized  as  vital  to  a curriculum  in  the  K-12  school. 

Longitudinal  studies  that  follow  students  for  a prolonged  period  of  time  to  see  how 
much  influence  the  depth  and  breadth  of  the  programs  exerts. 

Measured  data  about  the  impact  of  arts  knowledge  and  skills  on  other  subject  areas, 
standardized  tests,  graduation  rates,  and  critical  thinking  ability. 

Gathering  better  evidence  of  the  effect  of  learning  in  the  arts  on  student  cognition  and 
focus.  Not  “the  arts  improve  literacy”  but  “the  arts  improve  students’  ability  to  learn.” 

In  addition,  respondents  identified  valuing  the  arts  as  a need  of  the  field,  both  intrinsically  and  with 
respect  to  the  effect  of  arts  exposure  on  other  factors  (e.g.,  student  learning  in  other  subjects,  critical 
thinking,  self-esteem).  Some  respondents  agreed  with  their  district  and  school  staff  counterparts  on 
the  need  to  value  the  arts  as  a legitimate  discipline  on  par  with  other  academic  content  areas. 
Comments  regarding  the  value  of  the  arts  included: 

[We  need]  an  understanding  of  the  purpose  and  value  of  the  arts  not  just  as  a discipline 
in  itself  but  how  the  intrinsic  nature  of  the  arts  can  enhance  non-arts  classroom  learning 
and  also  be  supportive  of  arts  education. 

The  relationship  between  the  arts  and  ways  of  thinking  such  as  inquiry  and  questioning, 
problem  solving,  shifting  between  multiple  perspectives,  identity  formation,  and 
understandings  of  cultural  processes,  aesthetic  systems,  and  worldview  perspectives. 

Unfortunately,  in  the  U.S.,  we  don’t  broadly  value  any  learning  that  we  do  not  assess 
systematically.  This  tragic  fact  has  undermined  arts  education  for  decades,  especially 
beyond  K-5.  To  assess  arts  learning  is  to  make  sure  students  receive  this  essential 
education  in  more  districts,  more  states,  more  often. 

Office  of  education  staff  were  much  more  likely  to  express  a need  for  statewide  and/or  high-stakes 
assessment  in  the  arts,  along  with  increased  accountability.  More  than  twelve  percent  of  office  of 
education  staff  expressed  this  need  compared  to  less  than  three  percent  of  arts  council  staff, 
arts/cultural  organization  staff  and  researchers/evaluators.  Respondent  comments  included: 

Accountability.  While  people  balk  at  NCLB,  it  caused  change.  School  leaders  have  been 
shaking  in  their  boots  for  the  past  few  years  because  they  know  they  are  accountable  for 
achievement  results  and  failure  to  perform  has  major  consequences.  Where  is  that 
accountability  in  relation  to  the  arts?  We  need  that  fear!  Until  states  and  schools  are 
held  accountable  for  teaching  the  arts  as  core,  nothing  will  change. 

My  vision  would  be  mandatory'  reporting  about  how  a state  provides  instruction  and 
assesses  the  arts,  and  the  arts  need  to  be  a part  of  adequate  yearly  progress.  II  states  do 
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not  provide  this  data,  they  will  lose  funding.  If  they  do  provide  the  data,  they  will  have  a 
funding  source  dedicated  to  this  work. 

All  four  arts  - dance,  music,  theater  and  visual  arts  - need  to  be  assessed  nationwide  at 
the  elementary,  middle  and  high  school  levels,  to  ensure  instruction  and  student 
achievement. 

Exhibit  52  presents  the  qualitative  data  for  office  of  education  staff,  arts  council  staff,  arts/cultural 
organization  staff,  and  researchers/evaluators  regarding  making  the  case  for  the  arts  and  arts 
assessment. 


Exhibit  52  - Making  the  Case 


State/County 

State/County 

Arts/Cultural 

Arts 

Office  of 

Office  Arts 

Organization 

Researchers/ 

Education  Staff 

Council  Staff 

Staff 

Evaluators 

n % 

n % 

n % 

n % 

Value  of  arts 

11 

16.7 

10 

13.5 

100 

16.2 

31 

18.6 

Statewide/high-stakes  testing 

8 

12.1 

2 

2.7 

9 

1.5 

2 

1.2 

Data  management 

6 

9.1 

1 

1.4 

74 

12.0 

0 

0.0 

Research 

6 

9.1 

16 

21.6 

149 

24.1 

39 

23.4 

Support 

2 

3.0 

0 

0.0 

29 

4.7 

7 

4.2 

Advocacy 

0 

0.0 

0 

0.0 

18 

2.9 

0 

0.0 

Office  of  education  staff  n = 66,  arts  council  staff  n = 74,  arts/cultural  organization  staff  n = 618, 
researchers/evaluators  n = 167 


ADDITIONAL  NEEDS 

While  several  additional  needs  related  to  arts  assessment  were  identified  through  the  survey,  the 
overwhelming  additional  need  expressed  across  the  four  groups  of  respondents  was  funding. 
Respondents  indicated  needing  funding  for  several  arts  assessment-related  activities,  including: 

Funding  for  arts  assessment  itself  and  [for]  training  at  the  senior  staff  level  about  the 
importance  [of  arts  assessment].  It  is  something  that  you  have  to  spend  money  on  to  do 
properly. 

We  have  developed  [an  assessment  program  that]  tests  fourth  grade  art  and  music  with 
two  forms  of  a 45-item  selected  response  test  and  performance  tasks.  We  also  have  an 
intermediate  level  art  and  music  test  which  we  have  field-tested,  and  an  entry-level  dance 
and  theater  test.  The  later  have  been  suspended  due  to  funding  issues. 

We  need  funding  in  order  to  assess  the  degree  to  which  students  are  meeting  state 
standards  in  the  fine  arts.  It  is  important  that  all  states  have  access  to  these  resources  for 
measuring  outcomes. 

Funding  for  organizations  to  properly  assess  students’  knowledge  and  skill  set.  Our 
organization  is  currently  limited  in  what  we  can  assess  due  to  the  cost. 
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In  addition  to  funding,  some  respondents  identified  other  needs  related  to  arts  assessment,  including 
technology,  time,  and  the  ability  to  provide  access  to  all  students  and  meet  the  needs  of  diverse 
student  populations.  One  respondent  characterized  the  latter  as: 

How  do  I include  students  with  special  needs  in  an  inclusive  classroom  in  terms  of  their 
atypical  artistic  development  and  assess  them  by  acknowledging  their  unique  strengths 
and  weaknesses  as  a characterisdc  of  a neurological  learning  tvpe,  rather  than  focusing 
on  their  disruptive  behavior,  since  this  is  merely  a symptom  of  the  educational  system’s 
failure  to  address  their  learning  needs? 

Exhibit  53  presents  data  reflecting  respondents’  additional  needs  related  to  assessing  student 
learning  in  the  arts. 


Exhibit  53  - Additional  Needs 


State/County 

State/County 

Arts/Cultural 

Arts 

Office  of 

Office  Arts 

Organization 

Researchers/ 

Education  Staff 

Council  Staff 

Staff 

Evaluators 

n % 

n % 

n % 

n % 

Funding 

18 

27.3 

26 

35.1 

179 

29.0 

28 

16.8 

Diverse  students 

6 

9.1 

0 

0.0 

39 

6.3 

9 

5.4 

Technology 

3 

4.5 

4 

5.4 

9 

1.5 

12 

7.2 

None 

1 

1.5 

1 

1.4 

2 

0.3 

19 

11.4 

Anti-assessment 

0 

0.0 

1 

1.4 

22 

3.6 

10 

6.0 

Subjective  myth 

0 

0.0 

2 

2.7 

13 

2.1 

1 

0.6 

Time 

0 

0.0 

0 

0.0 

16 

2.6 

0 

0.0 

Office  of  education  staff  n = 66,  arts  council  staff  n — 74,  arts/cultural  organization  staff  n — 618, 
researchers/evaluators  n = 167 


OVERVIEW  OF  POLICYMAKER,  ARTS/CULTURAL  ORGNAIZATION  STAFF,  AND 
RESEARCHER/EVALUATOR  NEEDS 

In  addition  to  separating  the  qualitative  codes  into  the  above  categories,  this  secdon  provides  an 
overview  of  all  the  needs  of  the  field  expressed  by  office  of  education  staff,  arts  council  staff, 
arts/cultural  organization  staff,  and  researchers/evaluators.  As  noted  above,  various  types  of 
guidance  were  the  most  often  cited  need,  specifically  in  terms  of  a framework  to  guide  instruction 
and  assessment,  professional  development,  and  examples  of  high-quality  assessment  tools.  Exhibit 
54  presents  the  number  of  respondents  who  reported  the  following  as  needs  of  the  field. 
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Exhibit  54  — Overview  of  the  Needs  of  the  Field 


State/County  State/County  Arts/Cultural  Arts 

Office  of  Office  Arts  Organization  Researchers/ 

Education  Staff  Council  Staff  Staff  Evaluators 


n 

% 

n 

% 

n 

% 

n 

% 

Framework 

26 

39.4 

27 

36.5 

127 

20.6 

61 

36.5 

Professional  development 

22 

33.3 

33 

44.6 

142 

23.0 

37 

22.2 

Alternative  assessments 

19 

28.8 

12 

16.2 

119 

19.3 

40 

24.0 

Funding 

18 

27.3 

26 

35.1 

179 

29.0 

28 

16.8 

Exemplar  tools 

17 

25.8 

20 

27.0 

203 

32.8 

45 

26.9 

Value  of  arts 

11 

16.7 

10 

13.5 

100 

16.2 

31 

18.6 

Models 

10 

15.2 

14 

18.9 

80 

12.9 

27 

16.2 

Statewide/high-stakes  testing 

8 

12.1 

2 

2.7 

9 

1.5 

2 

1.2 

Resources 

7 

10.6 

7 

9.5 

84 

13.6 

6 

3.6 

Research 

6 

9.1 

16 

21.6 

149 

24.1 

39 

23.4 

Professional  learning  communities 

6 

9.1 

7 

9.5 

55 

8.9 

25 

15.0 

Certified  arts  educators 

6 

9.1 

4 

5.4 

26 

4.2 

3 

1.8 

Data  management 

6 

9.1 

1 

1.4 

74 

12.0 

0 

0.0 

Diverse  students 

6 

9.1 

0 

0.0 

39 

6.3 

9 

5.4 

Formative  assessment 

6 

9.1 

0 

0.0 

7 

1.1 

3 

1.8 

University  training 

5 

7.6 

4 

5.4 

11 

1.8 

12 

7.2 

Technology 

3 

4.5 

4 

5.4 

9 

1.5 

12 

7.2 

Support 

2 

3.0 

0 

0.0 

29 

4.7 

7 

4.2 

None 

1 

1.5 

1 

1.4 

2 

0.3 

19 

11.4 

Policy 

0 

0.0 

6 

8.1 

7 

1.1 

10 

6.0 

Subjective  myth 

0 

0.0 

2 

2.7 

13 

2.1 

1 

0.6 

Anti-assessment 

0 

0.0 

1 

1.4 

22 

3.6 

10 

6.0 

Advocacy 

0 

0.0 

0 

0.0 

18 

2.9 

0 

0.0 

Time 

0 

0.0 

0 

0.0 

16 

2.6 

0 

0.0 

Office  of  education  staff  n = 66,  arts  council  staff  n = 74,  arts/cultural  organization  staff  n = 618, 
researchers/ evaluators  n = 1 67 
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Summary  of  Perspectives  on  Student 
Assessment  in  the  Arts 


Looking  across  all  six  groups  of  respondents,  several  patterns  and  themes  emerged.  The 
overwhelming  majority  reported  having  some  experience  with  the  assessment  of  student  learning  in 
the  arts,  and  irom  their  responses  the  following  key  findings  emerged: 

Respondents  have  Multiple  Roles  and  their  Assessment  Experiences  are  Aligned  with  those 
Roles  — Office  of  education  staid,  district  staff,  and  researchers/evaluators  reported  more  varied 
types  of  experiences  with  arts  assessment,  such  as  conducting  professional  development  related  to 
arts  assessment,  establishing  arts  assessment  policies,  and  designing  tools  and  resources  for  use  by 
others.  Arts/cultural  staff  and  school  staff,  who  are  the  primary  providers  of  arts  instruction, 
reported  the  fewest  types  of  experiences  in  assessing  student  learning  in  the  arts. 

Professional  Development  is  the  Primary  Source  of  Training  on  Assessment  of  Student 
Learning  — In  general,  more  than  half  of  respondents  across  all  groups  reported  they  received 
training  on  assessing  student  knowledge  and  skills  in  the  arts  via  professional  development 
workshops  or  conferences.  Fewer  than  half  of  all  respondents,  including  fewer  than  one-quarter  for 
some  respondent  groups,  reported  receiving  undergraduate-  or  graduate-level  training  on  assessing 
student  learning.  Two  implications  stem  from  this  finding:  (1)  professional  development  plays  a key 
role  in  preparing  educators  and  others  to  assess  student  learning,  and  (2)  universities  need  to  include 
coursework  on  assessing  student  learning. 

A Wide  Variety  of  Assessment  Tools  are  Used  — For  the  types  of  knowledge  assessments  used, 
office  of  education  staff,  district  staff,  and  school  staff  were  more  likely  to  use  a teacher-  or  artist- 
developed  test,  while  arts  council  staff,  arts/cultural  organization  staff,  and  researchers/evaluators 
were  more  likely  to  use  a survey  administered  to  the  teacher  or  artist  to  gauge  student  knowledge. 

All  groups  of  respondents  reported  using  many  different  types  of  skills  assessment  tools,  including 
rubrics,  observation  protocols,  portfolio  reviews,  and  performance-based  assessments.  In  general, 
arts/cultural  organization  staff  reported  the  lowest  levels  oi  assessment  use. 

Respondents  were  Most  Likely  to  use  Skills  Assessment  Tools  Developed  by  Teachers  and 
Teaching  Artists  — For  the  assessment  of  student  skills , the  use  of  tools  developed  by  a teacher  or 
teaching  artist  was  the  most  common  response  across  all  six  respondent  groups.  The  majority  of 
office  of  education  staff,  district  staff,  and  school  staff  also  reported  using  teacher/ teaching  artist- 
developed  tools  for  the  assessment  of  student  knowledge  in  the  arts.  In  contrast,  the  most  common 
response  from  arts  council  staff,  arts/ cultural  organization  staff,  and  researchers/ evaluators  was  the 
use  of  externally  developed  knowledge  assessments. 

Respondent  Groups  use  Assessment  Data  for  Different  Purposes  — More  than  half  the  district 
and  school  staff  reported  using  data  to  as  part  of  a student  grade  and  to  provide  formative  feedback 
to  students.  In  contrast,  arts  council  staff,  arts/cultural  organization  staff,  and  researchers/evaluators 
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were  much  more  likely  to  indicate  they  used  data  for  program  evaluation  and  program/lesson 
improvement  purposes.  Further,  arts  council  and  arts/cultural  organization  staff  were  much  more 
likely  to  report  collecting  data  due  to  a funding  requirement  than  any  other  respondent  group. 

Lack  of  Clarity  between  Knowledge  and  Skills  — Respondents  across  all  groups  reported 
measuring  student  knowledge  with  methods  more  appropriate  for  measuring  skills,  and  vice  versa. 
This  indicated  respondents  may  not  clearly  distinguish  between  knowledge  and  skills,  and  calls  into 
question  to  the  validity  of  assessments  designed  by  those  who  may  not  fully  differentiate  knowledge 
from  skills. 

There  is  no  Single  Method  or  Place  Used  to  Locate  Assessment  Tools  — Respondents  across 
all  groups  employed  myriad  methods  when  they  needed  to  assess  student  learning  in  the  arts, 
particularly  modifying  tools  they  previously  used,  creating  new  ones,  and  searching  for  existing  tools 
— primarily  via  large  Internet  search  engines  like  Google  and  Yahoo.  In  addition,  survey 
respondents  identified  the  need  for  exemplar  tools  (e.g.,  specific  assessment  tools,  examples,  and 
item  banks)  as  key  for  the  arts  assessment  field.  This  indicates  a need  for  a single  location  or 
clearinghouse  for  measures,  scales,  and  item  banks  that  are  easily  accessed  and  vetted  to  ensure  high 
quality. 

The  Needs  of  the  Field  Centered  on  Four  Primary  Categories:  Guidance,  Trained 
Professional,  Making  the  Case,  and  Additional  Needs  — Guidance  was  needed  around  a clear 
arts  assessment  framework  that  aligns  standards,  curriculum,  and  assessment;  access  to  sample 
exemplar  tools  and  assessment  models;  high-quality  resources;  and  professional  learning 
communities  to  use  as  a sounding  board.  The  need  for  trained  professionals  instructing  and 
assessing  the  arts  included  additional  professional  development,  certification/licensure  in  all  art 
forms,  and  university-level  training.  Making  the  case  involved  demonstrating  the  value  and 
importance  of  the  art;  garnering  support  among  the  administrators,  leaders,  and  the  community;  and 
implementing  statewide  or  high-stakes  testing.  Additional  needs  of  the  field  included  funding,  time, 
technology,  meeting  the  needs  of  diverse  students,  and  overcoming  the  notion  that  the  arts  can  or 
should  not  be  assessed. 


Page  86 


ULIestEd  J 


Conclusion 


Page  87 


UUestEEd  J 


Conclusion 


Through  this  project,  the  NEA  and  WestEd  sought  to  collect,  analyze,  and  report  on  information 
about  current  practices  and  needs  of  the  field  related  to  the  assessment  of  K-12  student  learning  in 
the  arts.  Several  key  findings  emerged  from  the  study,  including  the  lack  of  publicly  available  high- 
quality  assessment  materials,  the  need  for  exemplars  and  models  of  assessment,  misconceptions 
about  arts  assessment,  and  the  need  for  professional  development  to  move  the  field  forward. 

It  is  important  to  note  that  this  study  — the  first  of  its  kind  in  the  arts  — contains  certain  limitations. 
With  regards  to  the  literature  review,  the  study  team  did  not  aggressively  seek  out  unpublished 
documents,  instead  focusing  primarily  on  documents  that  were  publicly  available.  While  this 
method  may  lead  to  an  underestimation  of  the  work  being  done  in  assessment  of  student  knowledge 
and  skills  in  the  arts,  it  presents  a more  accurate  picture  of  the  documents  available  to  anyone 
seeking  information  or  resources  on  the  topic.  In  addition,  the  survey  respondents  cannot  be 
assumed  to  be  nationally  representative  of  their  respective  groups,  limiting  the  ability  to  generalize 
results  across  the  entire  population.  Despite  these  limitations,  clear  themes  and  findings  emerged 
from  both  the  literature  review  and  survey  data. 

There  is  a lack  of  publicly  available,  high-quality  assessment  tools,  how-to  resources, 
technical  reports,  and  informational  documents. 

• While  727  different  documents  were  identified  during  the  literature  search,  56.9% 
did  not  directly  address  the  assessment  of  K-12  learning  in  the  arts  and  therefore 
were  considered  not  relevant  to  the  study.  Among  the  relevant  documents, 
approximately  half  (50.2%)  did  not  meet  the  standards  for  quality. 

• More  information  and  research  is  available  on  learning  through  the  arts  than  learning 
in  the  arts. 

• The  high-quality  literature  identified  during  the  review  process  was  scattered  across 
many  websites,  journals  articles,  books,  resources,  reports,  and  other  documents. 

Often,  high-quality  literature  was  mixed  in  with  irrelevant  and  lesser-quality  items. 

Much  time  and  energy  is  wasted  sorting  through  Internet  websites  searching  for 
quality7  information,  tools,  and  resources. 

• Somewhat  more  high-quality  assessment  tools  were  applicable  to  the  high  school 
grades  than  lower  grades.  However,  this  is  partially  due  to  the  nationwide 
assessments  such  as  Advanced  Placement  and  International  Baccalaureate  that  are 
only  available  at  the  high  school  level. 

• In  general,  very  few  research  and  evaluation  reports  were  found  despite  the  large 
number  of  funding  streams  that  require  evaluation  as  a term  of  funding.  The  lack  of 
dissemination  of  such  reports  limits  the  successful  models  available  for  replication  by 
both  program  staff  and  evaluators,  and  the  accessibility  of  such  models  and 
resources  was  identified  as  a need  of  the  field  by  survey  respondents. 
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• Publicly  available  research  was  cited  as  a need  of  the  field  by  survey  respondents  who 
wanted  to  use  findings  to  demonstrate  the  importance  of  art  as  an  academic  subject 
and  called  for  additional  funding  for  the  arts  because  of  its  impact  on  both  the 
cognitive  and  affective  domains. 

• The  over-abundance  of  poor-quality  information  on  the  Internet  and  lack  of  a single 
go-to  location  for  useful  resources  and  tools  was  evident  during  the  literature  review 
process  and  confirmed  by  many  survey  respondents  who  indicated  a clearinghouse 
for  arts  assessment  was  a need  of  the  field. 

There  is  a need  for  vetted,  high-quality  assessment  tools  and  models. 

• Two-thirds  of  relevant  assessment  tools  did  not  meet  the  standards  for  quality, 
leaving  only  30  assessment  tools  that  were  rated  as  both  relevant  and  high-quality'. 
High-quality'  assessment  tools  were  generally  created  by  testing  agencies  and/or  state 
education  agencies  that  dedicated  the  funds  and  resources  necessary'  to  develop  such 
tools. 

• Two-thirds  of  relevant  collections  of  assessment  tools  met  the  standards  for  quality'. 
Larger  collections  often  have  lower-quality'  tools  mixed  in  with  higher-quality'  tools. 
Higher- quality  collections  were  generally  developed  by  textbook  publishers  or 
maintained  and  vetted  by  state  education  agencies  or  larger  school  districts. 

• Not  all  assessments  or  collections  of  assessments  are  vetted  for  quality.  Available 
assessments  should  be  used  with  caution  and  consideration  should  be  given  to  the 
process  websites  use  to  collect  and  review  assessment  tools. 

• Three-quarters  of  respondents  reported  using  Internet  search  engines  to  look  for 
assessment  tools,  often  with  little  success.  When  respondents  could  not  find 
assessment  tools,  they  often  worked  to  develop  their  own  tools.  Favored  assessment 
tools  were  usually  developed  by  a classroom  teacher  or  teaching  artist,  and  the 
majority  of  respondents  indicated  there  were  challenges  using  their  favored 
assessment  tool.  Survey  respondents  indicated  the  field  needs  high-quality  tools  and 
models  to  improve  the  assessment  of  student  learning. 

There  is  a lack  of  understanding  about  what  a rubric  is  and  how  to  use  one,  and  there  is  not 
always  a clear  distinction  between  knowledge  and  skills. 

• Many  assessment  tools  identified  during  the  literature  review  claimed  to  be  rubrics 
when  they  were  more  accurately  described  as  rating  sheets  or  checklists.  The  field 
needs  to  better  understand  the  qualities  of  a rubric,  how  to  identify  and/ or  develop  a 
high-quality  rubric,  and  how  to  use  a rubric  to  assess  student  skills.  In  addition, 
survey  respondents  indicated  professional  development  on  these  topics  was  a need 
of  the  field. 

• The  literature  review  found  many  assessment  tools  that  inappropriately  measured 
knowledge  and/or  skills,  such  as  a paper-pencil  test  for  measuring  student  skills  in 
the  arts.  This  theme  was  also  found  in  survey  data,  with  respondents  reporting  using 
tools  to  measure  skills  that  are  more  appropriate  for  measuring  knowledge  and  vice- 
versa.  The  reliabilitv  and  validity  of  assessment  tools  and  resulting  data  are  called 
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into  question  without  a clear  understanding  of  the  distinction  between  knowledge 
and  skills. 

Survey  respondents  use  a variety  of  assessment  tools  to  collect  data  for  multiple  purposes. 

• All  groups  of  respondents  reported  using  many  different  types  of  skills  assessment 
tools,  including  rubrics,  observation  protocols,  portfolio  reviews,  and  performance- 
based  assessments. 

• The  majority'  of  survey  respondents  reported  that  the  tool  they  found  most  useful 
was  created  by  a teacher  or  teaching  artist. 

• Reasons  for  collecting  data  included  formative  feedback,  program  evaluation,  and 
district/school  accountability'.  School  staff  most  often  reported  using  data  for 
student  grades,  while  arts  and  cultural  organizations  and  state/county'  arts  council 
staff  were  significantly  more  likely  to  collect  data  as  a funding  requirement. 

Professional  development  is  needed  to  improve  the  assessment  of  student  learning  in  the 
arts. 


• Professional  development  is  needed  to  clear  up  misconceptions  about  rubrics  and 
other  assessment  tools,  and  to  ensure  a clear  understanding  of  and  distinction 
between  knowledge  and  skills  in  the  arts. 

• Survey  respondents  reported  needing  additional  training  on  topics  such  as  locating 
and  identifying  valid  assessment  tools,  developing  assessment  tools,  using  rubrics 
and  other  alternative  assessments,  gaining  administrator  and  public  support,  working 
with  community7  members,  building  public  value  for  the  arts,  and  using  assessment  to 
demonstrate  the  importance  of  arts. 

• The  primary'  source  for  training  related  to  arts  assessment  is  professional 
development  workshops  and  conferences.  Fewer  than  half  of  respondents  across  all 
groups  reported  received  undergraduate-  or  graduate-level  training  on  arts 
assessment. 

• Survey  respondents  also  identified  a need  for  professional  learning  communides  or 
communities  of  practice  where  teachers,  teaching  ardsts,  arts  specialists,  and  other  art 
educators  can  share  ideas,  lessons,  tools,  and  other  information,  or  get  feedback 
from  other  arts  professionals  about  the  lessons  or  assessment  tools  they  have 
created. 

Survey  respondents  reported  needs  of  the  field  around  four  categories  - guidance,  trained 
professionals,  making  the  case,  and  additional  needs. 

• Guidance  is  needed  in  terms  of  a clear  framework  that  aligns  standards,  curriculum, 
and  instruction;  access  to  exemplar  tools;  models  of  assessment  practice;  resources; 
and  professional  learning  communities  to  share  knowledge  and  ask  questions. 
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• Respondents  see  a need  for  professional  development,  university  training,  and 
certification  programs  in  all  art  forms  to  improve  instruction  and  assessment  in  the 
arts. 

• Making  the  case  included  demonstrating  the  value  and  importance  of  the  arts, 
including  having  research  to  show  the  impact  of  arts  education;  garnering  support 
from  district  and  school  leaders;  and  implementing  statewide  or  high-stakes  testing 
as  a method  to  gain  support  for  the  arts  since  some  believe  that  such  testing  would 
put  the  arts  on  par  with  other  subject  areas  currently  being  tested  such  as  English 
language  arts,  mathematics,  and  science. 

• Additional  needs  of  the  field  included  funding,  time,  technology,  meeting  the  needs 
of  diverse  students,  overcoming  anti-assessment  sentiments,  and  moving  past  the 
notion  that  the  arts  are  subjective  and  cannot  be  assessed. 


IMPROVING  THE  ASSESSMENT  OF  STUDENT  LEARNING  IN  THE  ARTS 


Several  recommendations  for  improving  the  assessment  of  student  learning  in  the  arts  can  be  drawn 
from  the  findings  of  this  study. 

Assemble  a national  advisory  committee  to  bridge  assets  and  come  to  consensus  on  how  to 
improve  arts  assessment. 

• The  advisory  committee  should  include  members  from  all  stakeholder  groups, 
including  teachers,  teaching  artists,  district  staff,  policymakers,  arts/cultural 
organization  staff,  and  researchers. 

• The  advisory  committee  can  contribute  to  prioritizing  needs  and  formulating 
solutions  that  will  move  the  arts  assessment  field  forward. 

Develop  a clearinghouse  of  high-quality,  well-vetted  assessment  tools,  resources,  and 
informational  documents. 

• The  clearinghouse  can  be  conceptualized  by  the  advisor)7  committee.  Some  secdons 
of  the  clearinghouse  will  cut  across  art  forms,  such  as  defining  knowledge  and  skills 
in  the  arts,  while  other  sections  will  need  art  form-specific  areas,  such  as  exemplar 
assessment  tools. 

• The  clearinghouse  can  have  a homepage  that  links  to  common  topic  areas  or 
document  types,  such  as  separate  secdons  for  assessment  tools,  information,  or  how- 
to guides. 

• The  clearinghouse  can  have  an  option  for  users  to  upload  their  materials  for 
consideration  if  there  is  a system  in  place  to  vet  the  materials  prior  to  making  them 
publicly  accessible. 

• While  the  clearinghouse  could  take  years  to  fully  establish,  in  the  short-term  a 
website  could  be  designed  that  provides  informational  materials  or  professional 
development  programs,  such  as  highlighting  what  is  a rubric,  how  is  it  used,  and  how 
to  develop  a rubric.  Priority  topics  could  be  established  by  the  advisor)7  committee. 
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Establish  online  professional  learning  communities. 

• Professional  learning  communities  could  be  one  aspect  of  the  clearinghouse,  left  as 
its  own  entity,  or  started  as  a separate  entity  and  merged  with  the  clearinghouse  once 
both  are  more  fully  established. 

• The  site  could  maintain  special  communities  for  different  stakeholder  groups  such  as 
policymakers,  teachers,  teaching  artists,  district  administrators,  arts  council  staff, 
arts/cultural  organization  staff,  and  researchers. 

• Communities  could  also  be  established  across  content  or  topic  area,  such  as  a 
location  where  participants  can  upload  their  assessment  tool  and  receive  constructive 
feedback,  or  share  ideas  on  meeting  the  needs  of  diverse  students. 

Develop  a research  agenda  for  moving  forward. 

• Additional  research  is  needed  on  a variety  of  topics,  such  as  identifying  models  of 
successful  practices  in  different  settings,  and  demonstrating  how  learning  in  the  arts 
is  beneficial  to  students. 

• Priorities  for  the  research  agenda  can  be  defined  by  the  advisory  committee.  The 
research  agenda  should  be  made  public  so  researchers  can  respond  to  the  needs.  A 
site  should  be  established  that  makes  vetted,  high-quality  research  reports  available  to 
the  field  (with  brief  summaries  and  full  reports).  Such  a site  would  allow  researchers 
to  learn  from  each  other  and  provide  practitioners  with  the  research  they  need  to 
move  the  field  forward. 

Increase  opportunities  for  professional  development. 

• There  is  a strong  need  for  professional  development,  both  to  address 
misconceptions  and  to  improve  the  assessment  of  student  learning  in  the  arts. 
Professional  development,  be  it  through  posting  informational  documents  or  hosting 
webinars,  is  needed  to  clarify  the  distinction  between  knowledge  and  skills  in  the  arts. 
Further,  professional  development  is  needed  to  clearly  define  rubrics,  their 
characteristics,  how  to  locate/develop  them,  and  how  they  are  used  for  assessing 
learning. 

• Professional  development  is  needed  to  dispel  the  myth  that  the  arts  are  subjective 
and  thus  not  able  to  be  objectively  assessed.  Workshops  or  webinars  on  meaningful 
methods  of  assessment  should  be  provided  to  the  field. 

• Professional  development  needs  to  be  tailored  to  the  appropriate  stakeholder  groups. 
Determine  what  topics  should  be  addressed  directly  and  what  topics  could  be 
addressed  using  a train-the-trainer  model.  The  capacity  of  office  of  education  and 
arts  council  staff  to  provide  professional  development  could  be  drawn  upon  to 
disseminate  information  to  the  field. 

• Specific  topics  for  professional  development  could  be  prioritized  by  the  advisory 
committee.  Potential  topics  include  defining  knowledge  and  skills,  showcasing 
various  types  of  assessment  tools  and  how  they  are  best  used,  highlighting  successful 
assessment  practices  in  different  settings,  working  with  community  members, 
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building  the  value  and  importance  of  the  arts,  and  identifying/approaching  potential 
funding  sources. 

• Professional  development  activities  through  technology  such  as  webinars  or  in- 
person  through  regional  trainings/ conferences  would  benefit  the  field  by  providing 
current  information,  developing  common  understandings,  sharing  successful 
practices,  and  building  the  knowledge  and  skills  needed  to  implement  assessment  in 
the  arts. 
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Definition  of  Terms 


The  following  provides  definitions  of  terms  as  used  in  this  report.  They  may  or  may  not  represent 

traditional  definitions  of  the  terminology. 

Alternative  Assessments  - Alternative  assessments  provide  an  opportunity  for  students  to  create  a 
response  to  a question,  task,  or  assignment.  In  contrast,  with  traditional  assessments  students 
choose  a response  such  as  multiple-choice,  true/ false,  or  matching  questions.  Alternative 
assessments  can  include  performances,  oral  presentations,  exhibitions,  and  reviews  of  work 
compiled  into  a portfolio. 

Assessment  of  Student  Learning  - Assessment  of  student  learning  is  specifically  designed  to 
measure  student  knowledge  and  skills.  The  resulting  data  can  be  used  for  many  purposes, 
including  student  grades,  measuring  change  over  time,  informing  lesson/program 
improvements,  gauging  lesson/program  effectiveness,  or  comparing  two  or  more  groups. 
Assessment  of  student  learning  may  be  one  component  of  program  evaluation  (see  below). 

Authentic  Assessment  — A form  of  assessment,  often  based  on  real-world  tasks  or  expectations, 
that  stresses  application  rather  than  recall  and  measures  master}7  by  requiring  students  to  develop 
responses  rather  than  select  from  predetermined  options.  Performance-based  assessment  (see 
below)  is  a widely  used  form  of  authentic  assessment,  particularly  in  the  arts. 

Checklist  - An  instrument  on  which  the  rater  assesses  student  performance  for  each  criterion  using 
a simple  scale  without  necessarily  having  clear  criteria,  such  as  yes/no,  proficient/not  proficient, 
or  a five-point  scale  ranging  from  poor  to  excellent  without  clear  gradations  for  quality7. 

Computer-based  Assessments  — A method  of  assessment  that  uses  computers  or  similar 
electronic-based  technology7  to  measure  performance  on  particular  criteria  or  attributes.  For 
example,  in  the  music  field  software  has  been  developed  that  can  measure  the  accuracy  of  a 
student’s  performance  in  hitting  the  correct  notes  with  the  proper  tempo.  In  some  cases, 
knowledge  assessments  (e.g.,  multiple-choice  tests)  have  been  administered  electronically  using 
online  survey  technologies. 

Formative  Assessment  — Formative  assessment  is  a process  used  during  instruction  that  provides 
feedback  to  adjust  ongoing  teaching  and  learning  (Fleritage,  2010). 

Formative  Evaluation  — Formative  evaluation  is  typically  conducted  for  improvement  of  a 
program  or  product  (Scriven,  1991). 

Observation  Protocol  - A form/ tool  used  while  observing  a task,  performance,  or  activity  that 
outlines  specific  criteria  or  elements  to  look  for  during  the  observation.  The  protocol  may 
include  checklist  items  such  as  whether  a specific  element  was  observed,  or  Likert-type  ratings 
gauging  the  extent  to  which  the  element  was  observed. 

Paper-Pencil  Assessment  - Knowledge  quizzes  or  tests  that  are  usually  comprised  of  multiple- 
choice,  true/false,  matching,  fill-in-the-blank,  short-response,  and/or  long-response  questions. 

Performance-based  Assessment  — An  assessment  technique  that  uses  systematic  observation  and 
evaluation  to  measure  a student’s  skill  as  they  complete  an  activity7  or  produce  a product. 
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Portfolio  Review  — A method  of  assessment,  usually  intended  to  measure  skills,  that  evaluates 
student  performance  or  achievement  based  on  a collection  of  samples  of  student  work. 
Portfolios  (i.e.,  compilations  of  student  work)  can  be  either  hardcopies  or  electronic  collections. 
The  review  process  usually  entails  the  use  of  an  assessment  tool  such  as  a checklist  or  rubric. 

Program  Evaluation  - Evaluation  is  the  application  of  scientific  methods  to  assess  the  design, 
implementation,  improvement,  or  outcomes  of  a program  (Rossi  & Freeman,  1993).  The 
assessment  of  student  learning  may  be  one  component  of  a program  evaluation. 

r cs^caAjlAca. 

Rubric  - A rubric  is'a  form/tool  used  to  grade  student  work  that  identifies  clear  guidelines/criteria 
for  distinguishing  between  gradadons  of  quality.  A holistic  rubric  provides  an  overall  rating  for 
elements  of  quality  and  levels  of  performance  while  an  analytic  rubric  measures  work  by 
assigning  ratings  to  various  elements  of  a product  or  performance  (i.e.,  one  performance  may  be 
measured  on  pitch,  tone,  tempo,  breath  control,  and  accuracy). 

Summative  Assessment  - In  contrast  to  formative  assessment  (see  above),  summative  assessment 
is  designed  to  indicate  the  extent  of  a student’s  success  in  meeting  criteria  used  to  measure 
intended  learning  outcomes  and  is  generally  administered  at  the  end  of  a unit,  topic,  module, 
course,  or  program. 

Teacher/ Artist  Survey  — An  assessment  technique  that  asks  the  classroom  teacher  or  teaching 
artist  working  directly  with  the  students  to  gauge  progress  and  learning  of  the  students.  In  some 
cases  the  teacher/artist  may  be  asked  to  complete  a survey  for  each  individual  student  with 
whom  they  work,  while  others  use  a single  survey  to  gauge  general  impressions  of  change  for  the 
class  as  a whole.  However,  little  research  has  examined  the  correlation  between  teachers’ 
impressions  of  student  learning  and  actual  gains  in  student  learning. 

Traditional  Assessment  - Traditional  assessment  is  generally  used  to  measure  student  knowledge 
and  typically  asks  students  to  choose  a response  to  a question,  such  as  multiple-choice  or  true- 
false. 
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Literature  Review  Guidelines 


Once  documents  were  identified,  a two-step  review  process  was  used  to  select  materials  for  the 
content  analysis.  The  first  step  was  to  determine  the  relevance  of  the  document  to  the  study.  The 
intention  of  this  step  was  to  weed  out  documents  and  materials  that  were  not  directly  related  to  the 
assessment  of  student  learning  in  the  arts.  Relevance  ratings  were  divided  into  three  broad 
categories  of  low,  moderate,  and  high,  and  within  those  categories  numerical  ratings  were  given  to 
account  for  variance  within  those  categories.  Within  the  low  category,  ratings  between  1 and  3 were 
given,  within  moderate  ratings  of  4 to  6 were  given,  and  within  the  high  category  ratings  between  7 
and  10  were  given.  Exhibit  A1  provides  the  basic  criteria  that  were  used  to  help  determine  the 
numerical  ratings  within  the  low,  moderate,  and  high  usefulness  categories. 


Exhibit  A1  - Guidelines  forjudging  Relevance 


Low 

Range  1 - 3 

Moderate 
Range  4-6 

High 

Range  7 - 10 

-Document  is  off  topic 

-Document  may  address  student 

-Document  directly  addresses  student 

-Focus  is  learning  through  the  arts 

learning  with  a focus  on  other  topics 

knowledge  and/or  skills  in  the  arts 

-Study  does  not  include  student 
learning  in  the  arts 

-Study  includes  some  reference  to 
learning  in  the  arts 

-Purpose  of  the  document  is  clear 

-Assessment  does  not  measure  student 

-Assessment  measures  knowledge/ skill 

-Assessment  clearly  measure 

kjnowkd«^4^^nthearts^^^^^^ 

When  a document  was  deemed  relevant  to  the  study,  it  underwent  the  second  step  in  the  review 
process  - determining  the  overall  quality7  of  the  document.  As  with  initial  ratings  of  relevance, 
quality7  ratings  were  divided  into  three  broad  categories  (low,  moderate,  and  high)  and  numerical 
ratings  were  given  to  account  for  variance  within  the  categories.  Quality  guidelines  were  developed 
for  each  type  of  document  (e.g.,  assessment  tool,  technical  report,  resource)  because  of  variance  in 
the  purpose  and  content  expected  in  each  type  of  document.  The  following  exhibits  present  the 
guidelines  used  for  the  different  tvpes  of  documents. 
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Exhibit  A 2 - Guidelines  for  Assessment  Tools 


Low 

Range  1 - 3 

Moderate 
Range  4-6 

High 

Range  7 - 10 

-Focus  is  not  clear  or  student 

-Assessment  is  partially  focused  on 

-Assessment  is  clear,  focused,  and 

knowledge/skills  is  not  prominent 

student  knowledge/skills  in  the  arts 

well-constructed 

-Assessment  measures  something 

-Assessment  may  be  too  advanced 

-Assessment  has  high  face  validity 

other  than  intended 

for  intended  audience 

-Assessment  reliability  and  validity 

-No/low  face  validity 
-Assessment  is  too  brief  or  too 
basic 

-Measures  of  student 
knowledge/skills  are  qualitative 
-Assessment  is  poorly  constructed 
and  content  is  of  low  quality 

-Assessment  has  moderate  face 
validity 

-Assessment  has  moderate 
reliability  and/or  validity,  or 
reliability/validity  not  provided 
-Assessment  is  not  well- 
constructed  (e.g.,  too  many  factors 
in  each  rubric  cell)  but  content  has 
merit 

are  high 

Exhibit  A3  - 

Guidelines  for  Collections  of  Assessment  Tools 

Low 

Moderate 

High 

Range  1 - 3 

Range  4-6 

Range  7-10 

-Lesson  plans/curriculum  not 

-Lesson  plans/curriculum  only 

-Lesson  plans/curriculum  clearly 

related  to  student  learning  in  the 

partially  related  to  student  learning 

relate  to  student  learning  in  the 

arts 

in  the  arts 

arts 

-Focus  is  not  clear  or  student 

-Assessments  are  partially  focused 

-Assessments  are  clear,  focused, 

knowledge/skills  is  not  prominent 

on  student  knowledge/skills  in  the 

and  well-constructed 

-Assessments  measure  something 

arts 

-Assessments  have  high  face 

other  than  intended 

-Assessments  may  be  too  advanced 

validity 

-No/low  face  validity 

for  intended  audience 

-Assessments  reliability  and  validity 

-Assessments  are  too  brief  or  too 
basic 

-Measures  of  student 
knowledge/skills  are  qualitative 
-Assessments  are  poorly 
constructed  and  content  is  of  low 
quality 

-Assessments  have  moderate  face 
validity 

-Assessments  have  moderate 
reliability  and/or  validity,  or 
reliability/validity  not  provided 
-Assessments  are  not  well- 
constructed  (e.g.,  too  many  factors 
in  each  rubric  cell)  but  content  has 
merit 

are  high 
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Exhibit  A4  - Guidelines  for  Resources 


Low 

Range  1 - 3 

Moderate 
Range  4 - 6 

High 

Range  7 - 10 

-Document  does  not  provide  "how- 

-Document  contains  minimal  "how- 

-Content  is  clear,  focused,  and 

to"  information 

to''  information 

serves  as  a good  "how-to"  with 

-Focus  is  not  clear 

-Document  has  limited  or  multiple 

regard  to  assessment  of  student 

-Document  is  too  brief  or  too  basic 

foci 

learning  in  the  arts 

-Document  uses  too  much  jargon 

-Document  is  easy  to  read  but  may 

-Resource  instructions  are  easy  to 

-Website  simply  provides  links  to 

lack  sufficient  details 

follow  with  examples  as  needed 

other  websites  with  no  original  text 

-Resource  instructions  are  difficult 
to  follow  and/or  examples  are  not 
given  as  needed 

-Document  may  be  too  advanced 

for  intended  audience 

-Lots  of  jargon  is  used,  but  usually 

explained 

-Document  is  well-organized  and 
easy  to  read/understand  and 
follow 

Exhibit  A5  - 

Guidelines  for  Informational  Materials 

Low 

Range  1-3 

Moderate 
Range  4-6 

High 

Range  7 - 10 

-Focus  is  not  clear 
-Document  is  too  brief  or  too  basic 
-Document  uses  too  much  jargon 
-Website  simply  provides  links  to 
other  websites  with  no  original  text 

-Document  has  limited  or  multiple 
foci 

-Document  is  easy  to  read  but  may 

lack  sufficient  details 

-Document  may  be  too  advanced 

for  intended  audience 

-Lots  of  jargon  is  used,  but  usually 

explained 

-Content  is  clear,  focused,  and 
appropriate  to  intended  audience 
-Document  is  well-organized  and 
easy  to  read/understand 
-Document  provides  worthwhile 
information  regarding  assessing 
student  learning  in  the  arts 

Exhibit  A6  - 

Guidelines  for  Informational  Materials 

Low 

Range  1-3 

Moderate 
Range  4-6 

High 

Range  7-10 

-Study  does  not  relate  to  student 
learning  in  the  arts 
-Study  findings  are  not  clearly 
drawn  from  data 
-Measures  of  student 
knowledge/skills  are  qualitative 

-Report  contains  general  findings 
but  lacks  sufficient  detail 
-Assessments  have  moderate  face 
validity 

-Assessments  used  have  low-to- 
moderate  reliability/validity  (if 
given) 

-Findings  are  clearly  drawn  from 
the  data  and  statistics 
-Assessments  used  have  high  face 
validity 

-Assessments  used  have  high 
reliability/validity  (if  given) 
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OMB  CLEARANCE  PACKAGE  DRAFT  - DO  NC 


NEA  Assessment  Survey 


MSTRIBLTE 

(OMB  -3135-0127) 


la.  What  state  do  you  live  in? 

1 . Which  of  the  following  best  describes  your  role?  (Please  select  one) 

□ State/County  Office  of  Education  staff 

□ State/County  arts  council  staff 

□ School/District  staff 

□ Arts  organization  staff 

□ Cultural  organization  staff 

□ Arts  evaluator 

□ Arts  researcher/education  researcher 

□ Other  (please  specify): 

[NOTE  - If  respondent  selects  School/District  staff  for  Ql,  they  complete  Q 2 and  skip  Q3.  For 
any  other  response  option,  respondent  should  complete  Q3  and  not  Q2] 

2a)  What  is  the  name  of  the  school  district  you  work  for? 

2b)  What  is  your  primary  role?  (Please  select  one) 

□ Principal 

□ Classroom  teacher 

□ Arts  educator/specialist 

□ Other  school  staff  (please  specify): 

□ District  superintendent  or  assistant  superintendent 

□ District  arts  liaison/arts  coordinator 

□ Other  district  staff  (please  specify): 

[NOTE  - If  respondent  selects  Principal,  Teacher,  Arts  educator/ specialist  or  Other  School  Staff  for 
2b,  then  they  complete  2c,  and  2e— 2f.  If  respondent  selects  District  superintendent  or  assistant 
superintendent,  District  arts  liaison/arts  coordinator  or  Other  district  staff,  then  thev  complete  2d- 

2f-] 

2c)  What  grades  does  your  school  serve?  (Please  select  all  that  apply) 

□ Elementary  (K-6) 

□ Middle  school  (5-8) 

□ High  school  (9-12) 

2d)  What  grades  does  your  district  serve?  (Please  select  all  that  apply) 

□ Elementary  (K-6) 

□ Middle  school  (5-8) 

□ High  school  (9-12) 
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OMB  CLEARANCE  PACKAGE  DRAFT  - DO  NOT  DISTRIBUTE 

(OMB  - 3135-0127) 

2 e)  What  art  form(s)  docs  your  school  address?  If  district  staff,  please  respond  for  your 
district  as  a whole.  (Please  select  all  that  apply) 

□ Dance 

□ Folk  Arts 

□ Literary  Arts 

□ Media  Arts 

□ Music 

□ Musical  Theater 

□ Opera 

□ Theater 

□ Visual  Arts 

□ Other  (please  specify): 

2f)  Which  of  the  following  does  your  school  offer?  If  district  staff,  please  respond  for  your 
district  as  a whole.  (Please  select  all  that  apply) 

□ In-school  arts  instruction  by  a classroom  teacher 

□ In-school  arts  instruction  by  a certified  arts  educator/specialist 

□ Artist  residencics/artists  in  classrooms 

□ Arts  integration 

□ General  arts  instruction 

□ After-school  arts  instruction 

□ Publisher  developed  arts  curriculum 

□ Teacher/school  developed  arts  curriculum 

□ District  developed  arts  curriculum 

□ Teacher  professional  development  in  the  arts 

□ Arts  assessments 

□ Other  (please  specify): 

3a)  What  is  the  name  of  the  agency/organization  you  work  for?  (fill-in) 

3b)  What  is  your  primary  role  in  the  agency/organization?  (Please  select  one) 

□ Arts  organization  staff 

□ Cultural  organization  staff 

□ Institution  of  Higher  Education  staff 

□ Teaching  Artist/Art  instructor 

□ Evaluator 

□ Researcher 

□ State/County  arts  education  coordinator 

□ State/County  arts  council  staff 

□ Other  (please  specify): 
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3c)  What  art  form(s)  does  your  agency  address?  (Please  select  all  that  apply) 

□ General  arts  education  across  disciplines 

□ Dance 

□ Folk  Arts 

□ Literary  Arts 

□ Media  Arts 

□ Music 

□ Musical  Theater 

□ Opera 

□ Theater 

□ Visual  Arts 

□ Other  (please  specify): 


3d)  Which 
□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 


of  the  following  does  your  agency/organization  offer?  (Please  select  all  that  apply) 

Classroom  teacher  professional  development  in  the  arts 

Teaching  artist/arts  educator  professional  development 

Pre-service  teacher  training  in  arts  education 

Graduate  level  teacher  training  in  arts  education 

Artist  mentoring  of  teachers 

Classroom-based  arts  instruction 

Private  arts  instruction 

Arts  advocacy 

Provide  funding/grants  to  arts  organizations/schools/districts 

Artist  residencies/teaching  artists  in  classrooms 

General  arts  instruction 

After-school  arts  instruction 

After-school  arts  programs 

Summer  arts  programs  for  K- 1 2 students 

Develop  arts  policies 

Develop  arts  curriculum 

Develop/administer  arts  assessments 

Program  evaluation 

Arts  education  research 

Other  (please  specify): 


4a.  What  do  you  see  as  the  needs  of  the  field  specifically  related  to  the  assessment  of  student 

knowledge  and  skills  in  the  arts? 


4b.  What  do  you  see  as  the  broader  needs  of  the  field  regarding  assessment  of  student  learning 
in  the  arts? 


5a.  What  assistance  do  you  (or  your  organization)  specifically  need  regarding  assessment  of 
student  knowledge  and  skills  in  the  arts? 

5b.  What  assistance  do  you  (or  your  organization)  need  regarding  assessment  of  student  learning 
in  the  arts  in  a broader  sense? 
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The  following  questions  draw  a distinction  between  student  knowledge  and  skills  in  the  arts. 
Knowledge  refers  to  student  content  knowledge  such  as  history,  terminology',  recognition,  and 
cultural  relevance.  Example  - students  ' ability’  to  identif’  a particular  dance  step,  whether  or  not 
they  are  capable  of  performing  it.  Skills  refers  to  students’  actual  ability  to  perform  or  produce 
art.  Example  - how  well  students  perform  the  dance  step.  We  will  begin  with  questions 
regarding  student  knowledge. 

6.  What  is  your  experience  with  assessment  of  student  knowledge  in  the  arts?  (Please  select  all 
that  apply) 

□ No  experience  with  assessment  of  student  knowledge  in  the  arts  [skip  to  OK] 

□ Used  a teacher/ school-developed  arts  assessment  tool  with  students 

□ Used  a district-developed  arts  assessment  tool  with  students 

□ Used  an  externally  developed  arts  assessment  tool  with  students 

□ Used  an  agency-developed  arts  assessment  tool  with  one  or  more  of  my 
organization’s  programs 

□ Used  an  externally  developed  arts  assessment  tool  with  one  or  more  of  my 
organization's  programs 

□ Received  undergraduate  or  graduate-level  training  on  assessment  of  arts  knowledge 

□ Attended  professional  dcvelopment/workshops  on  assessment  of  student  knowledge 
in  the  arts 

□ Conducted  professional  development/workshops  on  assessment  of  student  knowledge 
in  the  arts 

□ Developed  policies  on  assessment  of  student  knowledge  in  the  arts 

□ Developed  arts  assessment  tools/rcsources  for  use  by  others 

□ Conducted  research  on  student  knowledge  in  the  arts 

□ Required  grantees  or  funded  projects  to  assess  student  arts  knowledge 

□ Other  (please  specify): 

7.  Which  of  the  following  types  of  assessments  have  you  used  to  measure  student  knowledge  in 
the  arts?  (Please  select  all  that  apply) 

□ Computer-based  testing  software  (please  specify): 

□ Non-paper/pencil  assessment  (please  specify/describc): 

□ Test  developed  by  my  school/organization 

□ Test  included  with  textbook  or  lesson  plans 

□ Test  from  my  state 

□ Test  from  my  district 

□ Test  developed  by  evaluator  or  external  consultant 

□ Test  found  on  the  Internet 

□ Test  purchased  from  testing  agency 

□ Test  developed  by  classroom  teacher/certified  arts  educator 

□ Teacher/artist  survey  of  student  knowledge 

□ Other  (please  specify): 
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The  following  questions  pertain  to  assessment  of  student  skills  in  the  arts. 

8.  What  is  your  experience  with  assessment  of  student  skills  in  the  arts?  (Please  select  all  that 
apply) 

□ No  experience  with  assessment  of  student  skills  in  the  arts  , skip  to  01 1 

□ Used  a teacher / school-developed  arts  assessment  tool  with  students 

□ Used  a district-developed  arts  assessment  tool  with  students 

□ Used  an  externally  developed  arts  assessment  tool  with  students 

□ Used  an  agency-developed  arts  assessment  tool  with  one  or  more  of  my 
organization’s  programs 

□ Used  an  externally  developed  arts  assessment  tool  with  one  or  more  of  my 
organization’s  programs 

□ Received  undergraduate  or  graduate-level  training  on  assessment  of  arts  skills 

□ Attended  professional  development/workshops  on  assessment  of  student  skills  in  the 
arts 

□ Conducted  professional  development/workshops  on  assessment  of  student  skills  in 
the  arts 

□ Developed  policies  on  assessment  of  student  skills  in  the  arts 

□ Developed  arts  assessment  tests  for  specific  lesson  plans  or  curricula 

□ Conducted  research  on  student  skills  in  the  arts 

□ Required  grantees  or  funded  projects  to  assess  student  art  skills 

□ Other  (please  specify): 

9.  Which  of  the  following  assessment  tools  and/or  strategics  have  you  used  to  measure  student 
skills  in  the  arts?  (Please  select  all  that  apply) 

□ Observation  protocol 

□ Performance-based  assessment 

□ Portfolio  review 

□ Computer  software 

□ Paper/pcncil  test 

□ Checklist(s) 

□ Rubric 

□ Student  self-assessment 

□ Teacher/Artist  survey 

□ Other  (please  specify): 

[If  respondent  answers  “No  experience”  for  both  Q6  and  Q8,  then  ask  only  Qll,  012.  Q26, 
Q27  and  Q28] 
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10.  For  each  of  the  skill  assessments  you  have  used,  please  indicate  how  they  were  developed: 
(Please  select  all  that  apply) 


Rubric 

Observation 

protocol 

Perfonnance- 
based  assessment 

Portfolio  review 

Computer 

software 

Paper/pencil  test 

Checklist(s) 

Student  self- 
assessment 

Teacher/Artist 

survey 

Insert  Other 
response  from 
Q9 

Developed  by  classroom  teacher 

Developed  by  certified  arts  educator 

Developed  by  my  school/agency 

Included  with  textbook/lesson  plans 

Developed  by  my  state 

Developed  by  my  district 

Developed  by  evaluator  or  external 
consultant 

Found  on  the  Internet 

Purchased  from  testing  agency 

Unknown/Don't  know 

1 1 . If  you  needed  to  assess  student  knowledge  or  skills  in  the  arts,  which  of  the  following 
would  you  do?  (Please  select  all  that  apply) 

□ Use  tool  previously  developed  by  me  or  my  agency 

□ Modify  existing  tool  previously  used  by  me  or  my  agency 

□ Create  or  develop  new  assessment  tool 

□ Search  for  existing  assessment  tool  used  by  other  agencies/teachers 

□ Seek  professional  development/workshops  on  arts  assessment 

□ Hire  someone  to  create  a new  assessment  tool 

□ Other  (please  specify): 

(Q12  only  asked  if  respondent  checks  “search  for  existing”  in  Qll) 

12.  Which  of  the  following  would  you  use  to  locate  an  existing  tool?  (Please  select  all  that 
apply) 

□ Search  engine  (e.g.  Yahoo,  Google) 

□ Websites  (please  specify): 

□ Contact  specific  agency  (please  specify): 

□ Contact  specific  person  (please  specify): 

□ Library 

□ Other  (please  specify): 
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13.  Please  complete  the  following  table  for  the  specific  knowledge  and  skills  assessments  your 
agency  has  dcvcloped/used/administercd.  Please  indicate  the  name  of  each  assessment,  then 
select  all  that  apply  for  assessment  type,  grade  level,  and  art  form(s)  included  in  each 
assessment. 


Assessment 

Type 

Grade  Level 

Art  Fonn(s) 

Assessment  title 

Knowledge 

Skills 

Elem  (K-6) 

Middle  (5-8) 

High  (9-12) 

Dance 

n 

o 

> 

3 

C/2 

Media  Arts 

Music 



Theater 

Visual  Arts 

| 

Other 

A) 

B) 

C) 

D) 

E) 

F) 

G) 

H) 

I) 



□ I have  never  used  or  administered  an  arts  knowledge  or  skills  assessment,  [skip  to  Q26\ 


13a.  Please  indicate  which  of  the  following  assessment  tools  you  consider  most  useful. 
13b.  Why  do  you  consider  this  tool  most  useful? 

Please  refer  to  the  assessment  you  found  as  most  useful  to  answer  the  next  few  questions. 

14.  Who  developed  the  assessment  tool?  (Please  select  one) 

□ Developed  by  classroom  teacher 

□ Developed  by  teaching  artist/arts  educator/arts  specialist 

□ Developed  by  my  school  or  agency 

□ Included  with  textbook  or  lesson  plans 

□ Developed  by  state 

□ Developed  by  school  district 

□ Developed  by  evaluator  or  external  consultant 

□ Found  on  the  Internet 

□ Developed  by  testing  agency 

□ Other  (please  specify): 


UUest 
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15. 


What  was  the  purpose  of  administering  this  assessment?  (Please  select  all  that  apply) 

□ Required  by  state 

□ Required  by  district 

□ Required  by  school 

□ Required  by  funder 

□ Program  improvement 

□ Program  evaluation 

□ Measure  student  progress/learning 
u Classroom  test/student  grade 

□ Other  (please  specify): 


16.  How  well  did  the  assessment  work  for  your  purposes?  (select  one) 


Not  at  all 

1 2 3 4 5 6 7 8 

1 7.  Were  there  any  issues  or  challenges  in  using  the  assessment? 

□ Yes  (please  specify): 

□ No 


18. 


Who  scored/graded  the  assessment?  (Please  select  all  that  apply) 

□ Student  graded  by  him/herself 

□ Student  graded  by  peer(s) 

□ Classroom  teacher 

□ The  state 

□ The  school  district 

□ Testing  agency 

□ District  scoring  committee/ panel 

□ Arts  agency  staff 

□ Teaching  artist/ arts  educator 

□ Evaluator 

□ Other  (please  specify): 


19. 


How  were  the  data  used?  (Please  select  all  that  apply) 

□ School  accountability 

□ District  accountability 

□ Reported  to  funding  agency 

□ Program/lesson  improvement 

□ Program  evaluation 

□ Contributed  to  larger  research  effort 

□ Student  grade 

□ Provided  formative  feedback  to  students 

□ Published  in  a journal 

□ Unknown 

□ Other  (please  specify): 


Extremely 
9 10 
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20.  I lave  you  or  has  your  agency  developed  an  assessment  tool? 

□ Yes 

□ No  [ skip  to  Q26] 

If  multiple  assessments  have  been  developed,  please  refer  to  the  most  extensive  or  widely  used  to 
answer  the  next  few  questions. 

21.  What  did  the  assessment  measure?  (Please  select  all  that  apply) 

□ Knowledge 

□ Skills 

22.  Would  you/your  agency  be  willing  to  share  the  tool(s)  with  others? 

□ Yes 

□ No 

23.  Did  you  use  any  resources  (e.g.  research  reports,  model  programs,  how-to  guides)  in  the 
development  process? 

□ Yes 

□ No  [skip  to  Q26] 

24.  Please  specify  the  resource(s)  used  and  indicate  how  useful  each  resource  was  in  the 
development  process  on  a scale  of  1 to  10  where  1 is  Not  at  all  useful  and  a 10  is  Extremely 
useful. 


Name  of  Resource 

How  usefu 

was  t 

re  resource?  (check  one) 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

25.  Are  there  any  more  recent  resources  you  have  found  useful  that  were  not  available  at  the 
time  your  assessment  was  developed? 

□ Yes  (please  specify): 

□ No 

26.  Do  you  have  any  assessment  tools,  reports,  or  resources  you  would  be  willing  to  share? 

□ Yes  ->  Please  send  attachments,  URLs,  or  other  information  to  arts@westcd.org 

□ No 
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27.  Any  additional  thoughts  or  comments  you  would  like  to  share? 


28. 

□ Please  check  this  box  if  you  are  NOT  willing  to  let  us  contact  you  again  for 
further  information. 


Those  are  all  the  questions  we  have.  Thank  you  very  much  for  your  participation. 
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Exhibit  C15  - Types  of  Assessments  Used  to  Measure  Student  Skills  in  the  Arts 
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Exhibit  C17  - Types  of  Assessments  Used  to  Measure  Student  Knowledge  in  the  Arts 
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Exhibit  C20  - Types  of  Assessments  Used  to  Measure  Student  Skills  in  the  Arts 


4-J 

Vh 


CD 


to 

to 

s? 

o 

O 

LO 

LO 

o 

o 

LO 

V. 

Q) 

JC. 

o 

4-i 

LO 

o 

rsi 

rsi 

LO 

LO 

03 

CD 

cr> 

00 

rsi 

LD 

00 

to 

u 

_3 

X. 

03 

< 

03 

(U 

to 

> 

LU 

01 

"O 

f- 

00 

LD 

no 

CD 

00 

LD 

LO 

cc 

c 

no 

no 

no 

rH 

nsl 

no 

03 

"ro 

t- 

c 

no 

00 

LD 

LD 

CD 

rsj 

o 

as 

O 

00 

o 

no 

rsi 

LO 

3 

'5 

00 

LO 

T— 1 

'sT 

rsi 

LD 

4-> 

ro 

Jt 

3 

(N 

03 

U 

E 

4-» 

*' — ^ 

03 

WO 

to 

00 

v_ 

£ 

no 

rsi 

00 

cr» 

LO 

no 

< 

O 

m 

LO 

no 

rsj 

rH 

> 

+-» 

c 

u 

\P 

os 

o 

CD 

CO 

rsi 

T— 1 

rsi 

3.4 

CO 

no 

00 

no 

CD 

3 

c 

LD 

00 

kD 

r- 

O 

3 

fc: 

u 

O 

03 

u 

4-* 

<D 

to 

WO 

4-* 

03 

tr 

o 

^r 

00 

no 

no 

4-* 

WO 

< 

rsi 

rsi 

rH 

rH 

rH 

rsi 

it 

03 

4-4 

a? 

no 

rsi 

o 

00 

8.9 

no 

no 

r\ 

LD 

rH 

rH 

00 

CD 

kD 

kD 

CD 

WO 

“5 

o 

j= 

no 

o 

rH 

58 

rH 

no 

rH 

u 

wo 

c 

o 

rH 

O 

CT> 

LO 

LD 

no 

LO 

5fc 

a? 

o 

T— 1 

CO 

rsi 

LO 

rsj 

LO 

03 

d 

LO 

LD 

rsi 

no 

4-* 

LO 

00 

00 

00 

T— 1 

LO 

LO 

CD 

4-4 

U 

Z 

4-* 

to 

o 

T— 1 

CD 

LD 

no 

5 

no 

rsi 

rsi 

> 

LD 

rsj 

O 

T — 1 

CD 

O 

4-4 

c 

H— 

C 

a? 

o 

LO 

o 

00 

rH 

rH 

o 

3 

o 

o 

CD 

CD 

t / 

rH 

no 

LD 

k_j 

rH 

O 

01 

u 

44 

u 

03 

oT 

£ 

U 

3 

4-» 

03 

o 

"D 

LU 

CD 

O 

T— 1 

00 

no 

LO 

rH 

4-» 

WO 

T— 1 

rsi 

rsi 

rH 

rH 

rsj 

c 

<d 

E 

00 

to 

CD 

to 

to 

"o 

03 

u 

“O 

CD 

O 

CD 

4-> 

to 

03 

-f— ' 

o 

Q. 

03 

_Q 

£ 

cd 

£ 

«t 

to 

CD 

c 

O 

i) 

u 

c 

> 

cd 

o 

to 

u 

c 

4-> 

a: 

r\ 

CD 

CD 

4_» 

ro 

•> 

E 

U 

+-» 

Q. 

t/> 

C 

cd 

to 

t_ 

o 

"o 

>4— 

t 

Q. 

E 

t_ 

CD 

CL 

32 

lj 

CU 

u 

_Q 

_Q 

a> 

o 

o 

03 

-C 

3 

O 

a. 

Cl 

u 

CL 

u 

cn 

“3  U ^ 


rsi 


rsi  LO 
H f\  L/i 


LO 


> 

CD 

> 


t» 

D 


,:>age  128  UUeSt 


• • 
u 

\n 

3 


o 


<u 

fcJD 

T3 

a 

o 


c 

o 

"d 

</S 

Vw 

O 

c 

<u 

£ 

CD 

CD 

V 

CD 

CD 

< 

<u 

.c 


i/»  u tj  *: 

t £ c 5 

< S " 1 


o 

v.  2 ra  !t 

J 3 N U 

E 3 C 5? 

3 £?  c 


'«  ^ '1  st 
S £ § 3 
« S o i« 

«/»  3 LI 

O 

u 


‘o  o 
<u  £ q>  *j 

*£  5 S re 
re  3 .—  u 

*Sg3 

W Hi 


& 

3.1 

rH 

ro 

rH 

7T 

o 

in 

LO 

rsi 

in 

ID 

LO 

rsi 

ID 

m 

r\i 

tT 

rH 

m 

ID 

o 

in 

C 

ro 

7f 

ro 

rH 

ro 

SP 

os 

00 

00 

rH 

rH 

LO 

rH 

00 

rsj 

ro 

ro 

ro 

rH 

rsi 

rH 

rH 

ro 

rH 

LD 

LO 

r- 

CM 

rsi 

OJ 

rsi 

£ 

in 

rH 

ro 

00 

ro 

a? 

VO 

ID 

00 

in 

r>- 

in 

00 

00 

LO 

ro 

d 

rH 

rH 

LO 

rsi 

i — i 

rsi 

c 

cr> 

OD 

30 

13 

14 

LO 

s? 

LO 

7T 

CO 

ID 

o 

VO 

00 

o 

LO 

o 

rH 

ID 

Tf 

in 

r — i 

rsi 

m 

rH 

rsi 

ID 

i — i 

rsi 

ro 

c 

LO 

rsj 

rsi 

O 

ID 

00 

O 

ID 

rsi 

rsi 

ro 

rsi 

in 

t-H 

ro 

in 

LO 

00 

co 

ro 

o 

o 

q 

*3- 

7f 

LO 

ro 

ID 

rsi 

00 

CTi 

00 

rsj 

Ch 

CD 

£ 

rsi 

ro 

rsi 

ro 

rH 

a? 

6.4 

ro 

in 

ro 

rsi 

rsi 

CD 

O 

1^* 

rH 

7T 

o 

in 

r- 

tT 

CD 

ro 

ID 

ro 

OD 

07 

07 

C 

CM 

ro 

ro 

rH 

rsi 

rH 

00 


rs 


o ! 


-t- 

o 


in 

•+ 

ri 


T3 

o 


V 

05 

te 

O 

u 

C 

<u 

•n 

<u  , 

6 

w 


csi 

U 

3 

2 

x 

w 


ai 

CUD 

73 

_qj 

5 

o 

c 

-X 

c 

o 

CUD 

c 

c 

'to 


QJ 


QJ  03 
U =3 

F "D 

CD  03 
c- 

! <D  00 

Q.  Jr 

X o 
^ -a 
qj  03 

CUD  *- 

■O  5? 

QJ 


<D 

5 -g 
c 3 

-x  73 
QJ 

TO  QJ 
O QJ 
Z Cd 


TO 

QJ 

CUD 

73 

_QJ 

5 

o 

c 


-2  ? 


c 

o 

■*-» 

c 

a» 

E 

CL 

_o 

QJ 

> 

QJ 

73 

"to 

c 

o 


CO 

TO 

QJ 

CUD 

73 

QJ  QJ 
CUD 

o ]3 

-5  5 

c o 
o c 


<u 

*4— 

o 

c “• 

QJ  73 

E ^ 

i/i  C 

qj  a; 

CO  4— • 

co 
TO 


co  QJ 
QJ  CUD 
U 73 
QJ 


JX  13  — 
V4-  O 5 

O CO  o 

4-  9>  C 


QJ  co 
> co 
QJ  QJ 


£ U 

2 ^ 
CL  CL 
73  73 
QJ  QJ 


QJ 

CUD 

73 

_QJ 

5 

O 

c 


cJ  co 


co  to 
TO  QJ 


< U Q 


73  73 
QJ  QJ 

12 
QJ  73 
> C 
QJ  O 
O U 


C 

QJ 

‘u- 

QJ  L 

q-5 

x r 

aj 

QJ  L 

CUD 
73  I 
_QJ 

5| 

O 

C 


CUD 
73  tl\ 
QJ  TO  | 

'5  QJ  [ 
CT 

dJ  ^ , 

cc  O 


o 

Q 

I 


, 3 

73 


Page  129  ULIeSt'  J 


Exhibit  C22  - Types  of  Assessments  Used  to  Measure  Student  Knowledge  in  the  Arts 
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Exhibit  C30  - Types  of  Assessments  Used  to  Measure  Student  Skills  in  the  Arts 
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